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EXECUTIVE  SUMMARY 


The  ability  to  rapidly  understand,  adapt,  and  achieve  goals  in  cross-cultural  interactions  is  a 
critical  skill  for  those  engaged  in  various  forms  of  irregular  warfare.  Military  services, 
particularly  the  US  Anny  and  Marine  Corps,  desire  to  detennine  and  better  understand  the 
culture-general  skills  and  traits  needed  to  conduct  successful  cross-cultural  interactions  across 
unforeseen  cultures  in  order  to  improve  their  training  and  selection  process.  Once  these  skills  are 
identified,  policies  and  procedures  can  be  implemented  to  better  assess  and  select  as  well  as  train 
servicemembers  to  improve  the  likelihood  of  mission  success.  If  these  skills  are  not  trainable, 
there  is  the  possibility  of  selecting  people  based  on  their  aptitude  on  these  skills  or  based  on  traits 
they  possess  that  correlate  with  these  skills. 

The  goal  of  this  study  was  to  better  understand  the  culture-general  skills  and  traits  needed  to 
conduct  successful  cross-cultural  interactions  when  lacking  prior  in-depth  knowledge  of  the 
other  culture.  JHU/APL  conceptualized  the  study  in  three  phases:  (1)  a  behavioral  study  with 
active-duty  and  recently  retired  active-duty  military;  (2)  a  functional  neuroimaging  study  with  a 
college-student  population;  and  (3)  data  analyses  of  a  historical  dataset  on  US  Anny  Special 
Forces  Assessment  and  Selection  candidates.  The  second  phase  was  designed  to  immediately 
follow  the  first  phase,  but  regulatory  hurdles  prevented  any  data  from  being  collected  in  the  first 
two  phases,  so  results  are  only  available  for  Phase  3. 

Study  1.  In  the  first  phase  of  this  project,  an  experiment  was  designed  to  detect  cross-cultural 
adaptation  expertise  and  to  compare  and  contrast  expert  and  non-expert  traits  and  skills.  After  the 
experiment  was  designed,  obtained  local  institutional  review  board  approval,  and  developed  in 
preparation  to  run  participants,  ONR  then  ultimately  determined  that  this  project  should  not 
pursue  human-subject  testing.  This  study  was  designed  to  be  a  behavioral  study  designed  for 
active-duty  military  and  recently-retired  military.  This  study  would  have  been  conducted  online 
to  maximize  the  number  of  personnel  who  could  volunteer.  This  study  would  have  asked  about  a 
range  of  behavioral  and  personality  characteristics  including  intellect  (general  fluid  intelligence 
or  GFI)  and  the  ability  to  make  sense  of  complicated  situations  with  a  cultural  component  to  the 
complicated  situation,  referred  to  as  cultural  sensemaking;  it  would  also  have  asked  about  what 
participants  believed  were  the  ideal  skills  to  interact  with  people  from  other  cultures.  An 
important  aspect  of  this  first  study  was  to  have  obtained  data  about  who  succeeds  at  adapting 
across  cultures.  There  is  currently  no  operational  record  (or  empirical  data)  about  who  succeeds 
at  cross-cultural  interactions;  as  a  result,  military  services  are  unable  to  assess  and  select  for  or 
train  these  skills.  In  order  to  obtain  objective  data  about  these  skills,  participants  would  have 
been  allowed  to  nominate  people  who  they  felt  best  personified  being  adroit  at  cross-cultural 
interactions.  With  these  nominated  individuals,  JHU/APL  would  then  be  able  to  compare  their 
skills  and  traits  with  non-nominated  individuals  to  better  determine  those  skills  and  traits  that 
differentiate  experts  from  non-experts. 
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After  obtaining  local  Institutional  Review  Board  (IRB)  approval  for  this  study,  JHU/APL 
identified  multiple  potential  stakeholders  and  discussed  this  study  with  them.  This  included 
stakeholders  in  the  Marines,  Navy  Special  Warfare  Command  (WARCOM),  Army  Research 
Laboratory  (ARL)  and  US  Army  John  F.  Kennedy  Special  Warfare  Center  and  School 
(JFKSWCS).  JHU/APL  received  an  endorsement  from  the  US  Marine  Corps  (USMC)  Center  for 
Advanced  Operational  Culture  Learning  (CAOCL),  whose  procedural  and  domain  expertise 
made  significant  contributions  to  the  study  design.  However,  ONR  ultimately  opted  not  to  pursue 
human-subjects  testing  in  this  project,  so  no  data  was  obtained. 

Study  2.  A  functional  magnetic  resonance  imaging  (fMRI)  study  was  planned  that  focused  on 
analyzing  skills  needed  for  cross-cultural  interactions  that  may  be  affected  when  under  intense 
pressure.  This  study  was  designed  to  examine  both  cross-cultural  non-verbal  communication 
ability  as  well  as  the  ability  to  regulate  moral  emotions.  Emotion  regulation  is  a  critical  skill  for 
mission  success,  but  is  not  amenable  to  testing  via  self-reports  due  to  cultural  norms  preventing 
honest  assessment  of  emotions  as  well  as  because  the  process  of  regulating  emotions  can  be 
below  conscious  awareness.  An  in-lab  fMRI  study  would  allow  different  regulatory  strategies  to 
be  determined,  though,  and  these  different  strategies  could  be  critical  to  performance  in  high- 
stress  environments.  To  examine  cross-cultural  non-verbal  communication  ability,  the  study 
utilized  a  custom-designed  variation  of  the  Reading  the  Mind  in  the  Eyes  (RMES)  task.  This  task 
would  be  performed  in  both  high  and  low-workload  conditions,  to  better  simulate  how 
performance  would  be  affected  in  the  high-stress  environments  military  personnel  have  to 
perform  in. 

Stimuli  for  this  experiment  were  developed  and  the  experimental  design  was  completed. 
However,  once  ONR  determined  that  this  project  should  not  pursue  human-subjects  testing, 
efforts  to  finalize  this  study  and  obtain  local  IRB  approval  ceased  and  the  project  switched  focus. 

Study  3.  After  ONR  determined  that  this  project  should  not  pursue  human-subject  testing, 
JHU/APL  re-examined  how  to  proceed.  A  historical,  de-identified  dataset  was  identified  that  met 
the  spirit  of  this  project,  and  because  the  dataset  was  de-identified  it  did  not  constitute  human 
subjects  research.  The  dataset  was  from  JFKSWCS  and  contained  the  metrics  and  outcomes  for 
6,112  soldiers  who  attempted  the  Special  Forces  Assessment  and  Selection  (SFAS)  course  in 
FY13  and  FY14.  The  unit  responsible  for  SFAS,  1st  Battalion,  1st  Special  Warfare  Training 
Group  (Airborne)  (1st  Bn,  1st  SWTG(A))  was  integral  to  the  design  of  this  study  and  their 
specific  desire  was  to  improve  their  SFAS  process  but  they  also  had  a  generalized  interest  in 
identifying  and  training  cultural  adaptability.  After  ONR  approved  of  JHU/APL  working  with 
JFKSWCS  to  examine  their  SFAS  dataset,  JHU/APL  used  machine-learning  techniques  to 
analyze  current  SFAS  process  and  communicated  its  findings  to  1st  Bn,  1st  SWTG(A)).  JHU/APL 
examined  around  40  variables  for  each  individual  ranging  from  purely  cognitive  measures  of  that 
individual  such  as  full-scale  IQ  score  to  purely  physical  measures  such  as  two-mile  run  time. 
JHU/APL  determined  that  SF  selection  was  based  on  just  two  variables  related  to  a  single  skill, 
land  navigation  ability.  SFAS  outcome  could  be  determined  just  based  on  these  two  variables 
without  any  loss  of  accuracy  compared  to  using  all  the  variables,  indicating  that  these  two 
variables  are  driving  the  selection  process. 
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The  Neural  Correlates  of  Cross-Cultural  Adaptation  (NC3A)  project  was  a  missed  opportunity 
for  both  ONR  (to  support)  and  JHU/APL  (to  execute)  scientifically  rigorous,  operationally 
relevant  psychological  research.  In  general,  there  is  still  a  need  to  have  expertise  classified  so 
that  skills  and  traits  that  separate  expert  performers  from  non-experts  can  be  better  discerned,  in 
order  to  aid  in  training  those  skills  or  selecting  for  those  traits. 

Looking  to  the  future,  there  are  still  benefits  to  the  military  in  proceeding  with  Study  1;  the 
experiment  is  designed  and  ready  to  be  conducted  once  the  regulatory  approvals  are  obtained.  A 
second  direction  is  to  focus  on  analyzing  data  from  the  Special  Forces;  not  just  from  their  SFAS 
course,  but  from  follow-up  performance  metrics  such  as  how  the  soldiers  perform  in  the  Special 
Forces  Qualification  Course  (SFQC). 
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1  INTRODUCTION  AND  OVERVIEW 

1.1  INTRODUCTION 

Modern  warfare  is  an  inherently  uncertain  form  of  competition  (Boyd,  1976)  where 
consequential  decisions  are  often  based  on  incomplete,  inaccurate,  or  even  contradictory 
information  (HQMC,  1997);  thus,  decision-making  is  the  crucial,  principal  human  factor  in 
warfare  (Krulak,  1999).  Decision-making,  particular  in  irregular  warfare,  often  requires 
interaction  with  persons  of  a  different  culture;  be  they  as  a  source  of  information,  an  individual 
whom  a  servicemember  is  advising,  or  non-combatants  who  will  endure  the  consequences  of  a 
decision.  Improving  the  capacity  for  individual  decision-making  in  combat  has  been  identified  as 
a  Department  of  Defense  (DOD)  operational  need  (HQMC,  2011),  as  well  as  the  ability  to 
interact  with  persons  of  another  culture  in  order  to  acquire  the  information  necessary  to  make 
such  decisions.  Hence,  there  is  considerable  civilian  and  military  interest  in  detennining  which 
individuals  are  best  suited  for  working  abroad.  Much  of  the  research  focuses  on  long-term 
acculturation,  which  is  most  applicable  when  migrating  to  one  particular  culture  (see,  for 
example,  Berry,  Poortinga,  Segall,  &  Dasen,  2002).  However,  for  certain  types  of  jobs, 
individuals  are  required  to  go  to  multiple  locations  and  spend  a  relatively  short  amount  of  time 
(on  the  order  of  months)  at  each  place  thereby  prohibiting  integration  and  familiarization  with 
that  native  culture.  Despite  the  constraints  of  the  length  of  time  within  each  cultural  environment, 
the  assignments  demand  extensive  and  often  high-stake  interactions  with  locals.  These 
circumstances  can  be  difficult  and  are  often  emotionally  charged  and  fragile  (Goodwin,  2005). 
JHU/APL  refers  to  the  ability  to  succeed  in  this  type  of  short-term  scenario  as  “cross-cultural 
adaptation.” 

For  in-extremis  missions  where  specific  cultural  and/or  linguistic  expertise  is  not  available  (if  not 
organic  to  a  unit),  general-culture  competency  and  skills  such  as  knowledge  about  the  typical 
schedule  and  food  preferences  are  sufficient.  However,  there  are  short-term  missions  for  which 
these  culture-general  skills  are  insufficient  and  culture-specific  skills  are  required  to  assure  that 
one’s  mission  is  accomplished.  For  example,  the  skills  required  for  a  Francophile  to  successfully 
immigrate  and  integrate  into  French  culture  is  very  different  than  the  skillset  required  to  work  in 
France  for  three  weeks,  work  in  Italy  for  two  months,  then  consult  in  China,  etc.  Living  and 
working  in  so  many  different  cultures  can  be  particularly  stressful,  as  one  never  has  a  chance  to 
acculturate  sufficiently  to  feel  comfortable  in  that  particular  culture.  It  also  requires  the  ability  to 
assimilate  information  quickly;  not  only  to  learn  about  the  culture  rapidly,  but  also  to  adjust 
one’s  knowledge  of  the  culture  as  one  interacts  with  people  from  that  culture. 

To  address  this  important  DOD  operational  need  to  understand  cross-cultural  adaptation, 
JHU/APL  was  funded  by  the  Office  of  Naval  Research  (ONR)  to  study  the  requirements  for 
successful  cross-cultural  interactions.  JHU/APL  began  by  evaluating  previous  research 
conducted  on  related  research,  such  as  cross-cultural  competence,  where  interviews  with  active- 
duty  military  were  conducted  and  synthesized.  After  the  literature  review,  an  initial  study  was 
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designed  to  determine  whether  experts  and  non-experts  differed  along  a  number  of  the 
dimensions  hypothesized  in  that  literature  and  by  JHU/APL.  A  second  study  was  planned  that 
would  build  on  the  first  study,  but  be  designed  to  be  a  laboratory  study  to  examine  performance 
for  particular  skills  that  are  hard  to  test  via  self-report  studies  as  well  as  when  under  heavier 
cognitive  workload.  Finally,  a  third  study  was  conducted  that  examined  the  assessment  and 
selection  process  of,  the  US  Army  Special  Forces,  a  community  whose  missions  require  them  to 
interact  with  people  from  many  different  cultures.  The  goal  of  this  study  was  to  better  understand 
the  current  selection  process  so  that  the  process  can  continue  to  improve. 

A  general  framework  for  cross-cultural  psychology  (Vaughn  &  Phillips,  2009)  has  identified 
three  major  requirements  for  successful  cross-cultural  interactions:  intrapersonal  competence, 
interpersonal  competence,  and  cultural  competence.  Intrapersonal  competence  refers  to  the 
knowledge  of  one’s  self  and  individual  mental  skills  such  as  self-control,  openness  to  new 
situations,  and  levels  of  intelligence.  Interpersonal  skills  focus  more  on  social  skills.  Of  note  is 
the  fact  that  it  has  been  clearly  demonstrated  that  those  who  lack  interpersonal  skills  within  a 
mono-cultural  situation  will  demonstrate  poor  cross-cultural  skills  as  well  (Bochner,  2003). 
Cultural  competence  includes  culture-specific  knowledge  such  as  proficiency  in  the  appropriate 
language  and  the  awareness  of  cultural  norms  and  taboos;  it  also  encompasses  more  universal 
abilities  such  as  adapting  well  in  changing  circumstances  and  making  sense  of  unfamiliar  social 
environments  (Vaughn,  2010). 

A  key  intrapersonal  skill  is  the  ability  to  regulate  one’s  own  emotions.  Most  relevant  for  military 
personnel,  who  are  often  forced  to  interact  with  powerful  but  difficult  others,  is  regulating  one’s 
own  moral  emotions,  such  as  anger  and  disgust  (Greene  &  Haidt,  2002;  Haidt,  2003).  For 
example,  military  personnel  have  often  had  to  interact  with  local  warlords,  who  locally  may  treat 
women  and  children  in  a  way  likely  to  offend  people  with  Western  values.  However,  because  the 
mission  may  rely  on  working  with  these  warlords,  these  military  personnel  often  have  to  control 
their  own  feelings  in  order  to  accomplish  the  overall  mission,  a  very  difficult  task.  In  fact,  this 
very  problem  has  been  reported  recently  with  the  revelation  that  two  Special  Forces  soldiers 
were  punished  for  assaulting  and  Afghan  police  officer  who  had  reportedly  abused  a  child 
(http://www.nytimes.com/2015/09/21/world/asia/us-soldiers-told-to-ignore-afghan-allies-abuse- 
of-boys.html?_r=0). 

In  terms  of  interpersonal  skills,  a  great  deal  of  attention  has  been  focused  over  the  past  fifteen 
years  on  social  mindreading  which  generally  refers  to  an  individual’s  ability  to  infer  the  internal 
mental  states  (e.g.,  worried,  afraid,  puzzled)  of  another  person  and  to  use  that  inferred  state  to 
predict  that  person’s  likely  actions.  Also  referred  to  in  the  literature  as  “theory  of  mind”  or 
“mentalizing,”  a  major  deficiency  in  social  mindreading  skill  is  believed  to  lie  at  the  heart  of  the 
social  deficits  of  those  suffering  from  autism  spectrum  disorder  (Apperly,  2012;  Baron-Cohen, 
Leslie,  &  Frith,  1985;  Frith  &  Frith,  2006).  It  has  been  postulated  that  mindreading  may  be  one 
of  the  most  important,  as  well  as  one  of  the  most  vulnerable,  of  all  interpersonal  social  skills 
(Adolphs,  2010). 
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Cultural  competence,  the  third  of  the  Vaughn  &  Phillips  (2009)  triad,  is  not  as  easily  reduced  to  a 
specific  skill.  However,  early  work  by  Sieck  and  his  colleagues  (2008)  focused  attention  on  the 
high-level  skill  of  sensemaking  as  originally  proposed  by  Klein  and  his  colleagues  (1993)  and 
refined  by  Weick  (1995).  Sensemaking  is  the  idea  that  people  interpret  a  situation  by  fitting 
incoming  information  into  a  frame  that  links  them  to  other  elements  (Klein,  Phillips,  Rail,  & 
Peluso,  2004).  This  is  related  to  the  notion  of  cognitive  flexibility  because  individuals  must  be 
able  to  hold  many  frames  in  mind  and  test  these  various  frames  against  incoming  information. 
The  Sieck  focus  on  cultural  sensemaking  (2008)  is  a  natural  fit  to  the  cultural  competence  leg  of 
the  triad  and  is  the  third  major  focus  of  this  study. 

Previous  research  regarding  cross-cultural  adaptation  is  based  on  self-reported  data  (either 
interviews  or  questionnaires)  to  discover  so-called  “critical  components”  of  culture-general 
adaptability.  However,  self-report  studies  have  significant  risks  that  often  cause  the  results  to 
have  low  reliability.  This  complicates  the  ability  to  clearly  identify  the  underlying  factors 
associated  with  success  or  failure  in  cross-cultural  interactions;  low  reliability  in  the  self-reported 
results  may  also  account  for  the  poor  agreement  across  these  studies  about  which  factors  are  the 
most  critical  for  cross-cultural  adaptability.  For  instance,  college  students  who  scored  high  in 
emotional  stability  and  cognitive  flexibility  reacted  more  positively  to  a  stressful  intercultural 
situation  relative  to  students  scoring  low  in  emotional  stability  and  cognitive  flexibility,  based  on 
the  multi-cultural  personality  questionnaire  (van  der  Zee,  van  Oudenhoven,  &  de  Grijs,  2004).  In 
a  study  of  peacekeepers,  Vanderpool  (2002)  reported  that  attention  to  interpersonal  relations  and 
being  open  to  new  experiences,  as  assessed  by  the  cross-cultural  adaptability  scale,  accounted  for 
the  greatest  variability  in  achievement  of  their  mission.  Ross  and  colleagues  (2010)  administered 
the  cross-cultural  competence  inventory  test  to  military  personnel  and  reported  that  cognitive 
flexibility,  cross-cultural  empathy,  openness,  and  a  willingness  to  engage  with  other  cultures 
were  the  key  factors  for  success  in  tasks  requiring  cross-cultural  adaptation.  Although  the 
specific  results  of  these  three  studies  were  not  identical,  cognitive  flexibility  and  openness  were 
commonly  identified  as  key  to  cross  cultural  competence. 

To  avoid  the  problems  associated  with  relying  solely  on  self-report  data  and  the  resulting 
metrics,  the  specific  psychological  instruments  and  tasks  used  in  this  project  focus  on  the 
objective  assessment  of  cross-cultural  adaptability.  To  this  end,  JHU/APL  intended  to  administer 
performance-based  tasks  to  experts  and  non-experts  in  cross-cultural  adaptation,  while  also 
administering  those  personality  questionnaires  used  in  the  previous  literature  in  order  to  test  their 
findings  that  certain  traits  differentiate  experts  from  non-experts.  These  tasks,  along  with  the 
necessary  procedures  for  conducting  them  and  analyzing  the  resultant  data,  are  discussed  in  more 
detail  below. 

In  order  to  obtain  the  necessary  regulatory  approvals  to  conduct  the  planned  experiments,  after 
obtaining  local  IRB  approval,  JHU/APL  met  with  and  discussed  the  research  aims  with 
numerous  teams  throughout  the  military  branches.  For  example,  JHU/APL  discussed  the 
research  with  the  US  Marine  Corps  Center  for  Advanced  Operational  Culture  Learning 
(CAOCL),  who  endorsed  the  research  and  committed  to  not  only  helping  to  obtain  the  regulatory 
approvals  necessary  for  the  project  but  also  supporting  the  execution  of  the  study.  JHU/APL  also 
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met  with  the  USMC  IRB  Chair  to  discuss  recruiting,  and  prepared  documentation  on  the 
recruitment  process.  JHU/APL  further  met  with  Marine  Corps  University  (MCU),  who  agreed  to 
serve  as  survey  sponsor  for  the  first  study.  Ultimately,  ONR  opted  not  to  submit  this  project  to 
HRPO  for  approval  to  engage  in  human-subjects  testing. 

In  attempting  to  obtain  the  regulatory  approvals,  JHU/APL  discussed  their  research  with  the 
Army  Research  Lab  (ARL)  and  with  personnel  at  the  John  F.  Kennedy  Special  Warfare  Center 
and  School  (JFKSWCS)  at  Fort  Bragg  who  are  tasked  with  training  Special  Forces  personnel. 
The  interest  from  JFKSWCS  was  particularly  well-matched  because  Special  Forces  (SF)  soldiers 
often  operate  in  environments  where  they  are  required  to  interact  with  locals,  but  are  unable  to 
prepare  in  advance  for  a  particular  culture  due  to  limited  or  no  advance  knowledge  of  which 
culture  that  would  be.  In  fact,  JFKSWCS  had  already  begun  investigating  improvements  to 
cross-cultural  interaction  ability  of  their  personnel;  as  part  of  their  Special  Forces  Assessment 
and  Selection  (SFAS)  process,  they  had  collected  data  on  a  range  of  traits  and  abilities,  both 
physical  and  psychological,  to  try  to  improve  their  selection  process  for  Special  Forces. 
Specifically,  in  FY2015  SFAS  administered  the  Cultural  Intelligence  Scale  (CQS):  a  20-item 
self-report  instrument  that  has  four  metacognitive  items,  six  cognitive  items,  five  motivation 
items,  and  five  behavioral  items  (Ang  et  ah,  2007).  Additionally,  Phase  1  of  the  Special  Forces 
Qualification  Course  (SFQC)  includes  training  on  Cross-Cultural  Communication;  this  module, 
required  of  both  officers  and  each  of  the  enlisted  pipelines,  may  provide  an  interesting  outcome 
measure  using  the  CQS  as  a  predictive  variable  and  should  be  a  consideration  for  future  research. 

During  the  project  period,  JFKSWCS  provided  JHU/APL  with  anonymized  data  to  analyze  in 
order  to  help  SF  better  understand  their  ongoing  training  process.  JHU/APL  was  able  to  ascertain 
that  the  focus  was  not  on  certain  psychological  variables  that  may  be  important  for  ultimate 
performance  of  cross-cultural  interactions,  but  on  two  particular  land  navigation  tasks  that 
combine  physical  ability  with  executive  functioning,  especially  the  planning  ability. 

1.2  OPERATIONAL  CONTEXT 

There  has  been  a  clear  operational  need  to  be  able  to  adapt  to  various  cultures  since  the  end  of 
the  Cold  War.  Currently,  there  are  numerous  flashpoints  around  the  world  where  the  US  military 
could  be  sent  to  operate:  places  as  diverse  as  East  Africa  (e.g.  Somalia),  North  Africa  (e.g. 
Libya),  the  Middle  East  (e.g.  Yemen),  Central  Asia  (e.g.  Afghanistan),  and  Eastern  Europe  (e.g. 
Ukraine).  In  all  of  these  places,  interactions  with  locals  are  important  in  order  to  obtain 
information,  work  with  counter-insurgents,  and  to  make  lasting  changes.  But  how  to  train  US 
forces  who  may  have  to  interact  with  African  warlords  one  month  and  then  Afghan  village  elders 
the  next?  There  is  not  enough  time  between  location  shifts  to  familiarize  soldiers  with  the 
different  cultures,  so  instead  either  the  soldiers  do  not  have  significant  interaction  with  the  locals 
or  the  interactions  are  not  effective  to  achieve  mission  goals;  hence,  the  need  to  find  the  culture- 
general  abilities  that  are  needed  to  succeed  in  these  complex  interactions. 

The  role  of  identifying  such  matters  for  the  US  Marine  Corps  falls  to  the  CAOCL  who  were  not 
only  interested  in  the  research  topics  focused  on  by  JHU/APL  as  part  of  the  NC3A  project,  but 
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were  active  supporters  of  it.  In  fact,  CAOCL  shared  a  draft  of  a  set  of  skill  metrics  that  they 
independently  determined  were  important  for  Marines  to  possess  when  interacting  with  other 
cultures.  Their  list  of  skills  overlapped  considerably  with  those  JHU/APL  planned  to  examine  in 
Study  1,  as  seen  in  Table  1.  “D”  indicates  skills  that  would  have  been  measured  directly  while 
“I”  indicates  indirectly  measured  skills. 


Table  1.  Intersection  of  CAOCL  Cultural  Skills  and  NC3A  Study  1  Measures 


NC3A  Phase  I  Measures 

CAOCL  Cultural  Skills 

Letter  Sets 

Calendar  Test 

Following  Directions 

Cultural  Sensemaking 

Multidimensional 

Questionnaire 

Short-Form  (MPQ-40) 

Emotion  Regulation 

Questionnaire  (ERQ) 

Emotional  Intelligence 

Scale  (EIS) 

Cultural  Intelligence  Scale 

(CQS) 

Need  for  Cognition  Scale 

(NFCS) 

Moral  Foundations 

Questionnaire  (MFQ) 

Moral  Vignettes 

Cultural  Sensemaking 

D 

Perspective  Taking 

I 

I 

I 

I 

Observation 

Maintaining  Tact  &  Bearing 

D 

D 

D 

D 

Cultural  Self-Awareness 

D 

D 

I 

Trouble  Recovery 

Suspending  Judgment 

I 

The  JHU/APL  study  design  focused  on  identifying  credible  experts  at  cross-cultural  interactions, 
a  difficult  skill  to  measure  in  a  way  useful  to  military  forces.  By  determining  the  skills  and  traits 
possessed  by  credible  experts,  the  study  aimed  to  identify  ecologically  valid  skills  and  traits  that 
help  determine  cross-cultural  adaptation  expertise.  Some  of  the  CAOCL  cultural  skills  would  be 
hard  to  determine  solely  from  a  behavioral  assessment,  however.  Consider  the  components 
involved  in  maintaining  tact  and  bearing,  for  example.  To  maintain  tact  in  difficult  situations 
requires  being  able  to  regulate  emotions  while  under  duress.  A  self-report  questionnaire  asked  in 
low-stress  environment  is  unlikely  to  predict  how  a  person  would  respond  in  a  high-stress 
situation.  Furthermore,  there  are  cultural  norms  in  the  military  that  lead  many  to  not  answer 
questions  about  their  emotions  forthrightly,  preventing  the  self-report  assessment  from  being 
accurate.  A  second  experiment  was  planned  to  better  assess  those  skills  and  traits  that  require 
more  in-depth  testing  than  could  be  elicited  just  from  a  behavioral  assessment. 

1.3  SCOPE  AND  CONSTRAINTS  OF  WORK  CONDUCTED 

JHU/APL  conducted  a  pilot  study  on  a  custom-designed  cultural  sensemaking  task,  and  then 
planned  to  conduct  two  experiments:  (1)  an  online  behavioral  assessment  conducted  with  active- 
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duty  military  personnel,  and  (2)  an  fMRI  study  conducted  with  college  students  who  would  also 
take  the  online  behavioral  assessment  from  the  first  experiment. 

As  these  experiments  would  be  conducted  on  human  subjects,  a  rigorous  process  had  to  be 
undertaken  to  make  sure  that  the  experiments  would  be  ethical  and  safe  for  the  subjects. 
Whenever  a  study  is  conducted  on  active-duty  military,  the  process  requires  not  only  Institutional 
Review  Board  (IRB)  approval  from  the  local  institute,  but  also  approval  from  a  Human 
Resources  Protection  Officer  (HRPO)  at  ONR.  Initially  the  focus  of  this  project  was  on 
obtaining  local  IRB  approval  from  Johns  Hopkins  University,  which  was  obtained  in  June  2013. 
After  this,  preparations  were  made  to  obtain  HRPO  approval.  In  addition  to  providing  the 
necessary  paperwork  to  ONR,  JHU/APL  met  with  various  other  stakeholders  in  the  military. 
This  included  the  USMC  IRB  Chair  who,  upon  review  of  the  materials  provided  did  not  detect 
any  scientific,  administrative,  or  operational  problems  with  the  design.  JHU/APL  also  met  with 
representatives  from  CAOCL,  who  provided  both  regulatory  and  procedural  guidance.  Despite 
the  support  of  CAOCL,  the  USMC  IRB  Chair,  and  the  Marine  Corps  University,  ONR  opted  not 
to  submit  this  project  to  the  HRPO  for  review. 

After  being  informed  that  ONR  had  decided  not  to  submit  the  IRB  approved  protocol  for  HRPO 
approval,  the  project  plan  had  to  be  revised  to  complete  the  task  without  use  of  human-subjects 
research,  while  still  working  within  the  scope  of  this  project.  JHU/APL,  in  coordination  with  an 
ONR  program  manager  (for  this  approval,  see  Appendix  D:  Approval  from  ONR  to  Conduct 
Study  3),  determined  that  studying  historical,  anonymized  data  from  the  Special  Forces  would  be 
relevant  because  of  the  steps  that  they  had  already  taken  to  improve  their  selection  process  in 
order  to  identify  skills  and  traits  for  better-qualified  cross-cultural  adaptable  personnel.  Thus  the 
NC3A  project  was  adapted  to  analyze  data  provided  to  JHU/APL  by  Special  Forces. 

2  STUDY  1 :  ONLINE  BEHAVIORAL  STUDY 

2.1  BACKGROUND,  OBJECTIVES,  AND  SCOPE 

The  goal  of  Study  1  was  to  determine  the  skills  and  traits  associated  with  better  ability  at  culture- 
general  cross-cultural  adaptation,  with  a  specific  focus  on  military  personnel.  The  study  would 
test  and  verify  some  of  these  traits  and  skills  that  previous  literature  indicated  may  be  relevant. 
Critically  though,  JHU/APL  would  also  obtain  a  measure  of  expert  and  non-expert  performance 
on  cross-cultural  adaptation. 

There  is  currently  no  clear,  objective  measure  of  cross-cultural  adaptation  ability,  rendering  it 
difficult  to  determine  an  individual’s  level  of  ability.  The  plan  was  to  utilize  peer  evaluation  and 
recommendation  to  the  extent  possible.  JHU/APL  would  ask  military  personnel  for 
recommendations  on  who  they  would  consider  adroit  at  cross-culture  interactions,  and  then  test 
the  skills  and  traits  of  those  people  nominated  as  experts,  while  also  soliciting  volunteers  who 
were  not  nominated  as  the  non-expert  comparison  group.  Comparisons  would  then  be  made  on 
expert  and  non-expert  characteristics. 
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In  order  to  perfonn  a  meaningful  analysis,  JHU/APL  needed  to  obtain  approximately  50  experts 
and  150  non-experts.  Therefore,  JHU/APL  designed  an  openly-accessible  internet  survey  that 
would  be  accessible  to  anybody  with  an  internet  connection,  but  would  only  recruit  those  from  a 
military  population  or  those  recently  retired  from  the  military  in  the  last  five  years  to  participate. 
Although  an  internet  survey  restricted  some  of  the  types  of  tests  that  JHU/APL  could  perform 
(for  example,  any  tests  that  utilized  reaction  time,  which  could  have  been  compromised  by 
internet  lag  times),  JHU/APL  determined  the  internet  survey  was  the  best  solution  to  obtain  the 
highest  participation  rate.  Figure  1  is  the  homepage  of  the  proposed  online  study. 


GpD  JOHNS  HOPKINS 

\S  APPLIED  PHYSICS  LABORATORY 


Cross-Cultural  Adaptation  Study 


The  Johns  Hopkrs  University  rs  conducting  an  online  research  study  of  cross-cultural  adaptability  We  want  to  understand  the 
skills  and  traits  that  make  it  easier  to  interact  with  people  of  different  cuRures  Qualified  volunteers  wfl  be  asked  to  log  in  to  a 
study  website  and  complete  a  survey  about  aspects  of  personal,  social,  and  operational  experience,  along  with  a  senes  of 
cognitive  tasks  The  study  will  take  less  than  two  hours  to  complete 

if  you  meet  the  following  criteria,  you  are  eligible  to  participate  in  the  study: 

•  Must  be  between  18  and  60  years  old 

•  Must  have  traveled  outside  the  United  States  at  least  once  in  the  last  five  years 

•  Must  have  access  to  a  computer  with  Internet  access  and  a  web  browser 


Figure  1 :  Homepage  of  Study  1 

As  for  the  online  study  itself,  JHU/APL  primarily  used  established  tests  taken  from  the  literature 
for  the  stimuli.  In  addition  to  these  established  tests,  JHU/APL  also  created  a  custom  cultural 
sensemaking  task  that  would  be  performed  online.  Cultural  sensemaking  is  one  of  the  big  three 
cultural  competences  that  JHU/APL  determined  were  critical  for  mission  success.  However, 
there  is  no  established  method  to  objectively  test  cultural  sensemaking  ability,  so  JHU/APL 
designed  a  test  of  cultural  sensemaking  ability. 
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2.2  DETERMINING  CROSS-CULTURAL  ADAPTATION  EXPERTS 

Interviews  with  military  personnel  who  have  served  abroad  reveal  that  being  able  to  easily  and 
quickly  adapt  to  unfamiliar  cultures  is  critical  for  mission  success.  However,  identifying  the 
skills  and  traits  necessary  to  easily  and  quickly  adapt  to  a  culture  are  far  trickier.  The  problem  is 
that  there  is  no  established  way  to  objectively  measure  cross-cultural  adaptation  ability.  Without 
that  measure,  there  is  no  way  to  empirically  test  which  skills  and  traits  significantly  affect  cross- 
cultural  adaptation  performance. 

The  primary  goal  of  this  project  was  to  determine  a  way  to  rectify  this  situation  by  identifying 
expert  skills  and  traits  that  could  be  objectively  analyzed  through  various  tests  that  were 
expected  to  correlate  with  performance.  The  proposed  JHU/APL  solution  required  the  use  of 
active-duty  military  to  test  hypotheses  on  the  actual  population  of  interest  rather  than  utilizing 
college  students  or  other  proxies.  But  even  with  active-duty  military  participants,  it  was  still 
necessary  to  determine  expertise  in  order  to  make  meaningful  comparisons. 

JHU/APL  designed  the  experiment  to  incorporate  a  snowball  sampling  procedure  in  order  to 
obtain  the  requisite  experts.  In  snowball  sampling,  a  participant  is  asked  and  encouraged  to 
recommend  others  to  also  participate  who  are  in  turn  then  invited  to  participate  in  the 
experiment.  For  this  study,  participants  would  have  been  asked  to  nominate  people  they  knew, 
and  whom  they  considered  the  best  at  cross-cultural  interactions;  i.e.,  the  person(s)  they  would 
turn  to  if  they  were  experiencing  problems.  The  nominees  would  then  be  asked  to  participate  in 
this  experiment  and  be  classified  as  experts.  Any  participants  not  nominated  would  then  be 
considered  non-experts.  Although  this  type  of  recruitment  process  is  unorthodox,  JHU/APL 
argued  it  was  the  best  way  to  determine  expertise,  and  this  recruitment  procedure  was  approved 
by  the  Johns  Hopkins  IRB. 

There  are  clear  advantages  and  disadvantages  to  this  technique.  The  disadvantage  is  that  this 
method  is  subjective  and  unverifiable  as  participants  could  easily  nominate  participants  who  are 
not  true  experts  due  to  the  initial  participant’s  limited  experience  or  exposure.  One  advantage  is 
that  the  people  who  are  nominated  are  considered  experts  by  their  own  peers,  and  expertise  is 
otherwise  quite  difficult  to  determine  with  the  already  difficult-to-recruit  military  population. 
This  type  of  nomination  process  provides  some  ecological  validity  that  the  results  of  this  study 
would  be  of  some  utility  to  actual  military  forces.  Second,  this  technique  prevents  the  circular 
reasoning  of  pre-selecting  ‘experts’  based  on  performance  on  certain  skills  and  traits  that  the 
researchers  believe  to  be  critical  for  expertise.  For  example,  in  this  project  it  has  been 
hypothesized  that  emotion  regulation  was  critical  for  successful  cross-cultural  adaptation.  To  test 
this  hypothesis,  experts  and  non-experts  would  be  compared  and  experts  would  be  examined  to 
see  if  they  score  higher  on  emotion  regulation  ability.  However,  if  expertise  was  determined 
based  on  the  ability  to  regulate  emotions,  then  clearly  the  result  would  be  that  people  who  scored 
higher  in  cross-cultural  adaptation  expertise  score  higher  in  emotion  regulation  ability.  But  this 
result  would  be  driven  by  the  way  experts  were  selected,  as  the  selection  process  was  not 
independent  of  the  hypothesis.  If  future  training  and  selection  then  focused  on  this  ability,  this 
could  very  well  have  detrimental  effects  on  military  preparedness  by  wasting  resources.  An 
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independent  sample  to  test  hypotheses  is  thus  highly  valued.  Finally,  this  technique  can  be  very 
effective  in  a  case  where  the  abilities  may  be  highly  variable.  A  person  who  was  successful  in 
Afghanistan  may  possess  different  skills  than  a  person  who  was  effective  in  Nigeria,  but  the 
military  does  not  have  the  luxury  to  only  select  on  those  skills  that  are  useful  for  a  given  country 
or  region.  Thus  for  this  project  to  have  any  veridical  implications,  both  sets  of  experts  should  be 
nominated  to  obtain  the  largest  variance  in  cross-cultural  adaptation  experts. 

2.2.1  Ideal  Cross-Cultural  Adapter  Characteristics 

All  participants,  regardless  of  whether  they  nominate  somebody  or  not,  could  complete  a  survey 
about  the  skills  and  traits  they  believe  are  necessary  for  cross-cultural  adaptation  expertise.  This 
particular  assessment  was  designed  to  elicit  the  knowledge,  skills,  and  traits  of  the  idealized 
cross-cultural  adapter.  Part  1  of  the  nominator  survey  included  1 1  questions  scored  along  a  5- 
Point  Likert  Scale  (Strongly  Disagree,  Disagree,  Neither  Agree  nor  Disagree,  Agree,  and 
Strongly  Agree)  that  addressed  the  nominee’s  experience  with  and  aptitude  for  cross  cultural 
adaptation.  The  second  part  of  the  survey  was  a  derivation  of  the  Multicultural  Personality 
Questionnaire  (MPQ-SF)  (described  in  section  2.5)  where  the  nominator  was  to  indicate  how  the 
items  apply  to  the  nominee. 

2.3  CULTURAL  SENSEMAKING  ABILITY 

Sieck,  Smith,  &  Rasmussen  (2008)  found  that  a  critical  differentiating  feature  between  experts 
and  novices  in  cross-cultural  situations  was  the  ability  to  engage  in  cultural  sensemaking.  This 
ability  to  index  content,  make  attributions,  and  then  select  schema  was  identified  as  a  key 
differentiator  in  determining  current  ability  at  cross-cultural  adaptability.  JHU/APL  worked  with 
Sieck  and  Rasmussen  to  develop  a  cultural  sensemaking  task. 

In  order  to  understand  what  is  really  occurring  in  a  complicated  cross-cultural  environment, 
JHU/APL  designed  a  scenario  situation  with  various  stages  (Sieck,  Smith,  &  Rasmussen,  2013) 
to  test  and  analyze.  The  first  stage  was  ‘Notice  the  Anomaly,’  and  was  designed  to  determine 
how  early  participants  detected  a  misunderstanding  was  occurring.  The  next  step  was  to  inquire 
as  to  the  causes  of  the  unexpected  event.  It  is  generally  accepted  that  experts  at  cross-cultural 
interactions  tend  to  inquire  more  and  identify  a  larger  range  of  possible  causes  than  do  non¬ 
experts. 

After  inquiring  as  to  the  causes,  experts  tend  to  consider  multiple  alternative  explanations  for  the 
cultural  surprise.  This  often  requires  inhibiting  prepotent,  stereotypical  responses  to  the  event. 
Experts  have  to  have  the  ability  to  suspend  judgment  until  there  is  sufficient  evidence  to 
determine  what  caused  the  cultural  surprise. 

In  the  study,  participants  were  required  to  read  two  vignettes  where  a  cross-cultural 
misunderstanding  occurs  (see 
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Appendix  B:  Cultural  Sensemaking  Vignettes).  JHU/APL  ultimately  selected  two  vignettes:  (1) 
Breakfast  at  the  Cafe  (A.  14)  where  a  tourist  returns  to  a  cafe  they  had  had  breakfast  in  the  day 
before,  but  the  price  was  considerably  different  than  it  had  been  previously,  and  (2)  Riding  the 
Bus,  a  more  military-focused  vignette  (A.20),  where  after  a  change  in  military  action  there  was 
an  unexpected,  precipitous  drop  in  children  taking  the  bus  to  school. 

After  reading  the  vignettes,  participants  were  asked  a  series  of  free-response  questions.  The  first 
question  was  what  they  would  do  next.  The  second  asked  them  to  explain  what  happened  in  the 
situations.  Participants  were  required  to  spend  at  least  3  minutes  on  each  task,  allowing  them  to 
spend  time  generating  multiple  kinds  of  explanations.  After  that,  participants  also  spent  3 
minutes  generating  questions  about  the  situations. 

After  the  free-response  questions,  a  series  of  multiple-choice  questions  were  asked  of  the 
participants.  The  first  question  was  to  identify  which  action  would  be  the  best  response  in  that 
particular  situation.  The  second  was  about  which  question  they  would  most  want  answered. 
Finally,  a  multiple-choice  question  asked  about  the  most  likely  explanation  for  the  situations. 
This  was  followed  by  an  assessment  of  the  level  of  fixation  of  the  participants  on  their 
explanations  as  assessed  by  having  the  participant  rate  the  probability  of  their  explanations  being 
correct  as  well  as  the  probability  of  each  of  the  other  potential  possibilities. 

In  order  to  determine  the  best  vignettes  to  use,  a  pilot  study  was  run  (after  obtaining  IRB  and 
HRPO  approval)  on  participants  via  Amazon  Mechanical  Turk.  The  cultural  sensemaking  task 
was  never  tested  with  a  military  population,  as  only  the  pilot  study  was  completed.  Thus  no 
results  were  obtained  about  if  performance  on  this  task  correlates  with  expertise  in  cross-cultural 
adaptation  that  will  be  of  interest  to  the  military  population. 

2.4  ASSESSING  PERSONALITY  TRAITS 

Personality  is  the  stable  set  of  characteristics  and  tendencies  that  determine  commonalities  and 
differences  in  people’s  thoughts,  feelings,  and  actions  that  have  continuity  over  time  and  are  not 
the  proximal  result  of  temporary  social  and/or  biological  pressures  (McWilliams,  2004). 
Understanding  an  individual’s  personality  provides  insight  as  to  how  they  attend  to  experience, 
modulate  affect,  view  him  or  herself,  uses  ideation,  manages  stress,  and  interacts  with  others 
(McWilliams,  2004),  and  thus  it  is  valuable  to  include  a  measure  of  personality  in  an  experiment 
concerned  with  not  only  interaction  with  others,  but  also  modulating  affect.  The  Five-Factor 
Model  (FFM),  or  Big  Five,  of  personality  consists  of  five  macrolevel  domains  or  dimensions; 
neuroticism,  extroversion,  openness,  agreeableness,  and  conscientiousness  (Costa  &  McCrae, 
1992).  The  FFM  is  a  widely  used  model  of  personality  that,  although  not  diagnostically  sensitive 
enough  for  clinical  use,  has  considerable  research  utility  (McCrae  &  John,  1992).  Neuroticism  is 
characterized  by  emotional  instability  and  a  susceptibility  to  psychological  distress  (Costa  & 
McCrae,  1992).  Neuroticism  negatively  correlates  with  performance  under  stress,  particularly 
when  the  stress  is  exceedingly  high  such  as  during  combat  (Grier,  2012).  Neurotic  individuals 
often  appear  anxious  and/or  moody  and  are  generally  less  likely  to  be  perceived  as  capable 
leaders  (Johnson  &  Hill,  2009).  Extroversion  is  an  outward  focus  of  one’s  psychological  energy 
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as  opposed  to  an  inward  focus  on  subjective  experience  or  Introversion  (Costa  &  McCrae,  1992). 
There  is  no  consistent  relationship  between  extraversion  and  performance  under  stress  beyond 
the  fact  that  extroverts  tend  to  perforin  better  with  others  around  (Grier,  2012)  and  generally 
exhibit  affiliate  and/or  sociable  behavior  (Johnson  &  Hill,  2009).  Openness  is  the  degree  to 
which  individuals  seek  “new  Information,”  often  manifesting  in  intellectual  curiosity,  creativity, 
and  a  general  non-conformity  (Costa  &  McCrae,  1992).  Openness  correlates  highly  with 
microlevel  traits  such  as  uncertainty  and  ambiguity  tolerance  (Johnson  &  Hill,  2009). 
Agreeableness  is  the  tendency  to  be  caring,  cooperative,  and  tolerant  (Costa  &  McCrae,  1992). 
Agreeable  individuals  tend  to  interact  well  with  others,  and  thus  tend  to  be  viewed  favorably  as 
leaders  whereas  disagreeable  individuals  come  across  as  rigid  and/or  oppositional  (Johnson  & 
Hill,  2009).  Conscientiousness  is  the  tendency  to  be  industrious,  organized,  and  responsible 
(Costa  &  McCrae,  1992).  Conscientious  individuals  are  generally  dependable  and  achievement- 
oriented;  attributes  that  tend  to  correlate  with  leadership  ability  (Johnson  &  Hill,  2009). 

Among  the  limitations  of  macrolevel  personality  traits  in  predicting  performance  in  military 
cohorts  is  the  lack  of  specificity  in  the  broad  constructs  (Grier,  2012)  as  well  as  the  vast  gamut  of 
experience  in  modern  military  operations  ranging  from  the  mundane  to  the  terrorizing  (Parmak, 
Mylle,  &  Euwema,  2013).  While  there  are  combinations  of  personality  traits  associated  with 
more  effective  leaders,  elevated  on  Extroversion,  Openness,  Agreeableness,  and  Conscientious 
and  a  correspondingly  low  score  on  Neuroticism,  (Johnson  &  Hill,  2009),  there  may  be 
additional  attributes  that  contribute  to  an  individual’s  ability  to  interact  with  others  from  a 
different  culture. 

Given  the  motivation  of  the  study,  it  was  determined  that  a  more  applied  measure  would  be 
preferable  to  a  generalized  personality  assessment,  and  thus  the  short  form  Multicultural 
Personality  Questionnaire  (MPQ-SF)  was  selected.  The  MPQ-SF  is  a  40-item  self-report 
measure  that  also  uses  a  five-factor  model  (van  der  Zee,  van  Oudenhoven,  Ponterotto,  &  Fietzer, 
2013).  The  five  factors  in  the  model  were  cultural  empathy,  open-mindedness,  emotional 
stability,  social  initiative  and  flexibility.  Cultural  empathy  refers  to  empathizing  with  individuals 
of  a  different  culture;  open-mindedness  refers  specifically  openness  toward  cultural  differences; 
emotional  stability  is  the  ability  to  remain  clam  in  the  face  of  cultural  uncertainty;  social 
initiative  refers  to  exercising  initiative  in  cross-cultural  social  settings;  and  flexibility  refers  to 
interpreting  novel  cultural  stimuli  and  adjusting  positively  (van  der  Zee  et  ah,  2013). 

2.5  REGULATORY  CHALLENGES  AND  LESSONS  LEARNED 

JHU/APL  faced  many  challenges  and  ultimately  learned  quite  a  few  lessons  over  the  course  of 
this  project.  The  first  study  was  to  be  conducted  using  active-duty  military  as  well  as  recently 
retired  military  personnel.  The  reason  for  this  was  to  ensure  that  any  results  would  actually  be  of 
utility  for  future  military  decisions  regarding  personnel  who  experience  unique  circumstances 
compared  to  others  in  having  to  adapt  to  random  cultures. 
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In  order  to  recruit  active-duty  personnel  for  the  study,  JHU/APL  also  had  to  navigate  through 
additional  regulatory  procedures  beyond  Johns  Hopkins  University  IRB  approval.  ONR 
graciously  provided  a  liaison  to  help  with  the  process. 

Knowing  that  having  active-duty  military  personnel  participate  in  the  study  would  be  a 
significant  challenge,  the  first  study  was  designed  to  minimize  physical  and  privacy  risks  for 
active-duty  military.  JHU/APL  obtained  local  IRB  approval  in  June,  2013.  The  ONR  HRPO 
liaison  estimated  at  that  time  it  would  take  about  two  additional  months  to  obtain  HRPO 
approval;  instead,  it  was  not  until  17  months  later  that  it  was  decided  this  project  would  not  be 
submitted  to  the  HRPO  for  review. 

During  the  interim,  significant  effort  was  spent  modifying  the  experiment  in  order  to  obtain  the 
necessary  HRPO  approval.  JHU/APL  worked  with  organizations  such  as  CAOCL  and  the 
Marines  Survey  Officer  searching  for  acceptable  solutions  to  move  the  study  forward.  Despite 
those  efforts,  ONR  opted  not  to  submit  this  study  to  the  HRPO  for  approval;  therefore,  no 
military-related  results  were  obtained  for  this  study. 

There  is  still  an  urgent  need  to  improve  understanding  of  cross-cultural  adaptation  ability; 
particularly  in  the  case  of  the  military  who  regularly  send  personnel  to  interact  with  warlords, 
sheikhs,  foreign  military,  foreign  doctors,  and  countless  others.  JHU/APL  could  have  focused 
their  recruitment  solely  on  recently  retired  military  personnel  (which  would  probably  not  have 
raised  so  many  regulatory  concerns)  or  college  students  (who  are  much  more  accessible  a 
population),  but  then  the  results  would  have  had  less  direct  implications  for  military  personnel. 
However,  given  the  time  and  resources  spent  on  attempting  to  obtain  HRPO  approval,  future 
studies  on  this  topic  may  be  better  served  by  not  having  active-duty  military  as  participants. 

2.6  DISCUSSION 

Study  1  was  designed  to  test  extant  hypotheses  about  what  skills  and  traits  make  a  person 
particularly  adroit  at  cross-cultural  interactions,  especially  in  a  culture-general  manner.  It  was 
designed  to  test  this  hypothesis  with  active-duty  military,  the  main  population  of  interest,  so  that 
the  results  could  immediately  impact  military  selection  and  training.  Unfortunately,  no  actual 
data  was  obtained  in  this  study  except  for  pilot  data  on  cultural  sensemaking  (which  was  not 
collected  on  active-duty  military,  but  only  on  Amazon  Mechanical  Turk  population),  due  to 
ONR  determining  this  project  should  not  proceed  to  the  human-subjects  testing  phases. 

Until  the  military  better  understands  the  culture-general  skills  and  traits  indicative  of  better 
cross-cultural  performance,  the  military  is  unable  to  train  these  skills  or  select  for  these  traits.  In 
the  near-term,  research  into  what  skills  and  traits  are  most  important  will  mostly  be  those 
extracted  from  interviews,  but  this  does  not  lend  itself  to  a  clear  way  to  test  the  findings  of  these 
various  researchers.  Hopefully,  future  studies  will  determine  a  way  that  can  technically  test  some 
of  these  hypotheses  while  successfully  obtaining  the  regulatory  approvals  needed  for  the  results 
to  be  of  utility  to  the  military. 
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3  STUDY  2:  FUNCTIONAL  NEUROIMAGING  STUDY 

3.1  BACKGROUND,  OBJECTIVES,  AND  SCOPE 

Study  2  was  designed  to  examine  in  depth  those  aspects  of  cross-cultural  adaptation  that  would 
be  difficult  to  measure  via  purely  behavioral  tests:  moral  emotion  regulation  and  cross-cultural 
empathy.  However,  this  study  was  planned  to  be  completed  after  Study  1,  allowing  participants 
in  Study  2  to  be  given  the  behavioral  test  administered  in  Study  1  to  determine  the  traits  and 
skills  of  participants  in  Study  2.  Due  to  the  Study  1  regulatory  hurdles  discussed  above,  this 
study  was  not  completed  and  thus  this  section  will  be  rather  brief.  The  design  and  logic  of  Study 
2,  as  well  as  some  of  the  stimuli  that  were  designed,  will  be  discussed, 

Study  2  was  designed  to  focus  on  moral  emotion  regulation  and  cross-cultural  empathy.  Emotion 
regulation  can  be  difficult  to  test  via  self-assessment  for  two  main  reasons:  first,  there  can  be  a 
cultural  norm  preventing  people  from  admitting  their  emotional  response  (a  norm  particularly 
common  with  military  personnel);  and  second,  many  are  not  consciously  aware  of  how  they 
regulate  emotions  even  though  the  method  by  which  they  do  so  can  have  critical  effects  in  how 
they  will  react  in  particularly  stressful  environments.  Fortunately,  functional  magnetic  resonance 
imaging  (fMRI)  can  determine  different  brain  regions  associated  with  different  strategies  of 
emotion  regulation  such  as  suppression  or  reappraisal. 

Moral  emotions  are  the  subset  of  emotions  that  have  a  moral  component  to  them  such  as  disgust 
and  anger.  Military  personnel,  in  particular,  often  have  to  deal  with  incredibly  difficult  people, 
such  as  warlords,  who  have  a  very  different  value  system  that  can  be  particularly  onerous  to  a 
Western  value  system.  Yet  these  personnel  have  to  overcome  their  emotions  to  work  with  these 
people  to  accomplish  the  mission.  The  ability  to  successfully  do  so  thus  seems  likely  to  be 
needed  for  long-term  mission  success. 

Cross-cultural  empathy  is  needed  for  non-verbal  communication.  The  ability  to  read  the 
emotions  of  another  person  can  greatly  facilitate  communication.  However,  this  sort  of  ability 
can  be  particularly  tricky  across  cultures.  For  example,  there  is  a  large  body  of  research  that  has 
determined  that  people  do  not  look  as  closely  at  the  faces  of  people  of  other  races,  which  is  part 
of  the  reason  that  people  of  other  races  look  more  alike  to  some.  Naturally  this  is  a  problem  for 
effective  cross-cultural  communication  and  adapting  to  unfamiliar  cultures. 

Behavioral  experiments  can  determine  a  person’s  reaction  time  and  accuracy  at  doing  some 
cross-cultural  empathy  tasks  such  as  the  cross-cultural  Reading  the  Mind  in  the  Eyes  (RMES) 
task  JHU/APL  planned  to  conduct.  RMES  was  developed  to  test  how  well  an  individual  could 
attune  to  the  mental  state  of  another  by  exposing  the  individual  to  25  photographs  of  the  eye- 
region  of  the  face  and  determining  which  of  four  words  best  describes  the  emotion  the  stimulus 
is  expressing  (Baron-Cohen  et  ah,  2001).  There  is  a  limitation  to  this,  though;  for  real-world 
military  personnel,  they  often  have  to  do  this  task  in  high-stress  environments  that  are  impossible 
to  ethically  simulate  in  a  lab.  It  is  thus  possible  that  performance  alone  wouldn’t  translate  well  to 
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how  a  person  would  react  in  a  high-stress  environment.  Instead,  it  could  be  that  the  method  used 
to  do  the  processing  in  the  RMES  task  may  better  simulate  how  people  would  perform  in  an 
extremely  high-stress  environment.  People  can  utilize  two  different  processing  systems  to 
perform  the  same  task.  System  1  processing  refers  to  fast,  associate  heuristics  people  often  use  to 
do  tasks,  and  these  heuristics  are  less  susceptible  to  interference  from  stress  or  workload.  System 
2  processing  refers  to  the  slower,  rule-based  processes  people  often  use  when  using  more  rational 
and  logical  tasks.  People  who  do  cross-cultural  empathy  via  System  1  processing  are  more  likely 
to  perform  better  in  high-stress  real-world  environments  than  those  who  utilize  System  2 
processing,  even  if  they  perform  no  better  in  lower-stress  environments. 

System  2  processing  may  be  more  accurate  than  System  1  in  a  lot  of  tasks,  even  if  somewhat 
slower.  Given  the  speed-accuracy  tradeoff,  it  can  be  difficult  to  determine  therefore  if  faster,  less 
accurate  performance  is  better,  worse,  or  equal  to  slower,  more  accurate  performance. 
Fortunately,  different  brain  regions  are  associated  with  System  1  and  System  2  processing,  so  it 
would  have  been  possible  to  determine  what  type  of  processing  a  person  was  engaged  in  while 
doing  the  cross-cultural  empathy  task. 

3.2  STIMULI  DEVELOPMENT  AND  PROCEDURE 

To  measure  cross-cultural  empathy,  JHU/APL  was  going  to  use  a  modified  version  of  the  RMES 
task.  Specifically,  each  participant  was  going  to  see  four  sets  of  eyes:  (1)  those  belonging  to 
people  of  the  same  culture  and  race  as  them  (either  Caucasian  or  African-American  eyes),  (2) 
familiar-other  (Caucasian  if  the  participant  was  African-American,  or  African-American  if  the 
participant  was  Caucasian),  (3)  East  Asian  eyes  (specifically  Japanese  eyes,  but  not  Japanese- 
American),  and  (4)  Arabic  eyes.  Participants  would  be  shown  the  eyes  and  would  have  to  choose 
the  emotion  that  they  felt  was  represented  by  the  eyes.  The  general  finding  is  that  participants  are 
better  than  they  expect  on  eyes  from  their  own  culture  and  race,  but  perfonn  worse  on  eyes  from 
another  race  and  another  culture. 

In  preparation  for  producing  the  cross-cultural  RMES  task,  JHU/APL:  (1)  collected  established 
expression  data  sets  covering  a  number  of  ethnicities  (Table  2),  across  a  variety  of  ethnicities 
(Adams  et  ah,  2010;  Alzeyara,  Sakr,  &  Ziadee,  2012;  Baron-Cohen  &  Wheelwright,  2001; 
Kaulard,  Cunningham,  Bulthoff,  &  Wallraven,  2012;  Lucey  et  ah,  2010),  (2)  began  the 
processing  to  normalize  the  images  for  presentation  under  fMRI,  (3)  began  developing  a 
standardized  vocabulary  for  target  and  distractor  expression  words  and  a  strategy  for  properly 
assembling  them  for  each  presented  image,  and  (4)  began  developing  the  code  to  execute  the 
dual  task. 


Table  2  -  Expression  Data  Sets  and  their  Ethnicities 


Expression  set 

Ethnicity 

Gender 

Color 

Gaze  direction 

NimStim 

African-American, 

Asian-American, 

Caucasian 

Both 

Color, 

Grayscale 

Head  on 
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Baron-Cohen 

Caucasian 

Both 

Grayscale 

Various 

Adams 

Asian 

Both 

Grayscale 

Various 

JAFFE 

Japanese 

Female 

Grayscale 

Head  on 

MSFDE 

African-American 

Both 

Grayscale 

Head  on 

PICS 

Iranian 

Female 

Color 

Head  on 

CK+ 

Asian,  Caucasian 

Both 

Grayscale 

Head  on 

Whereas  these  sets  may  capture  the  whole  face  or  provide  a  series  of  frames  showing  the 
expression  as  it  manifests,  or  are  in  color,  JHU/APL  prepared  methods  and  cropped  an 
equivalently  sized  and  isoluminant  eye  region  to  the  Baron-Cohen  set,  selected  the  most 
expressive  frame  from  a  series,  and  converted  the  images  to  grayscale.  In  addition,  JHU/APL 
began  to  generate  presentation  sets  balanced  for  gender  and  gaze  direction. 

Words  describing  emotions  are  presented  at  the  comer  of  each  image.  One  of  the  four  is  the 
target,  the  other  three  are  distractors.  The  distractors  must  be  selected  with  consideration  to  the 
valence  and  cluster  of  emotion  of  the  target. 

Figure  2  shows  an  example  of  RMES  stimulus  of  Asian  eyes  from  the  Adams  dataset. 
Participants  see  only  the  eyes  and  are  forced  to  guess  which  of  the  four  emotions  apply  to  those 
eyes.  In  this  example,  participants  who  selected  ‘worried’  would  be  correct. 


Figure  2:  Example  of  RMES  stimulus 

This  experiment  would  also  increase  the  participant’s  stress  level  by  sometimes  incorporating  a 
high-workload  N-back  task  (Figure  3).  In  an  N-back  task,  a  participant  has  to  determine  if  a 
stimuli  is  the  same  as  the  stimuli  that  occurred  N  trials  earlier,  requiring  a  participant  to 
continuously  update  their  working  memory  (WM)  and  delete  items  from  WM.  Participants 
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would  face  blocks  where  they  only  had  to  do  the  RMES  task,  whereas  other  blocks  where  they 
saw  the  eyes  but  only  had  to  say  if  the  emotion  in  the  eyes  was  the  same  as  they  saw  N  stimuli 
previously.  A  third  type  of  block  would  merge  these  two  into  a  high-workload  dual-task  where 
participants  on  each  stimuli  had  to  both  determine  the  emotion  of  the  eyes  as  well  as  determine  if 
it  was  the  same  emotion  they  had  selected  N  stimuli  previously. 


Figure  3:  Example  of  N-Back  Task.  With  0-back  the  participant  just  identifies  surprised  faces.  With  1-back, 
the  participant  has  to  determine  if  the  emotion  on  the  face  was  the  same  as  the  previous  face.  With  2-back, 
the  participant  determines  if  the  emotion  was  the  same  as  emotions  two  stimuli  earlier.  In  the  actual  stimuli, 

only  the  eyes  would  be  visible. 

The  moral  emotion  regulation  task  would  have  been  based  on  Moll  et  ah,  (2011) 
where  participants  read  short  vignettes  meant  to  trigger  different  moral  emotions  including  guilt 
and  disgust  (see  Figure  4).  Some  of  the  vignettes  that  were  planned  to  be  included  were  meant  to 
have  triggered  a  moral  emotion  in  people  from  one  culture  but  not  from  another.  In  collaboration 
with  a  Turkish  collaborator,  Aysecan  Boduroglu,  some  vignettes  were  identified  as  likely  to 
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trigger  moral  outrage  in  Western  participants,  but  unlikely  to  do  so  with  Turkish  participants. 
These  included  vignettes  about  dowries  and  child-brides,  common  in  many  parts  of  the  world  but 
something  that  can  trigger  disgust  in  Western  Participants.  Note  that  these  vignettes  were  not 
simplified  to  be  ideal  for  fMRI  studies,  but  they  are  still  included  in 
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Appendix  C:  Emotion  Regulation  .  It  was  also  planned  to  use  some  of  the  original  vignettes  used 
by  Moll  et  al. 


I  Uoilrtd  Nttratmar  54  mil)  ITJ5-1 7V 


1717 


Your  mom  called  you  and  sard  she  didn't  (cel  wed 
You  ignored  her,  and  the  r»e*1  day  she  d«d 

EMSUI  CULT  NEUTRAL 


Target 

Guilt 


You  had  breakfast  at  a  hotel  restaurant  The  waiter 
brought  the  check  and  you  billed  i  to  your  room 


Target 

Neutral 


You  talked  to  an  old  lady  on  a  bus.  She  told  you  how 
she  lost  her  husband  and  children  in  an  accident 

SMSARR  CXSGJST 


Target 


At  a  Itxmal  dinner,  someone  told  a  funny  joke 
While  laughing  you  suddenly  burped  loudly 


EMBARR 


Target 

Embarrassment 


Figure  4  -  Example  of  Moral  Emotion  Regulation  Task,  from  J  Moll,  Oliverira-Souza,  Bramati,  &  Grafman 

(2002) 

The  goal  was  to  determine  if  experts  were  better  at  regulating  moral  emotions  than  non-experts. 
It  was  also  hypothesized  that  experts  engage  in  cognitive  reappraisal  strategies  to  regulate 
emotions,  whereas  non-experts  were  more  likely  to  engage  in  emotion  suppression  strategies. 
Further,  JHU/APL  hypothesized  that  experts  may  experience  less  emotional  arousal  from  these 
vignettes  even  if  they  were  not  told  to  regulate  their  emotions.  By  testing  this,  it  could  be 
discerned  if  experts  were  better  at  regulating  emotions  or  were  just  less  susceptible  to  emotional 
feelings  in  the  first  place. 

3.3  DISCUSSION 

Study  2  was  designed  to  be  an  in-lab  study  letting  JHU/APL  study,  in  depth,  certain  moral 
emotion  regulation  and  cross-cultural  empathy.  These  two  facets  are  important  for  cross-cultural 
adaptation,  but  would  be  difficult  to  study  outside  of  the  lab.  The  study  was  designed  to  be  an 
fMRI  study  to  better  determine  information  that  could  not  be  elicited  from  the  behavioral  data 
only.  Specifically,  the  design  would  have  helped  to  determine  whether  participants  were  engaged 
in  System  1  or  System  2  processing  when  experiencing  cross-cultural  empathy  tasks  and  what 
type  of  emotion  regulation  strategy  participants  engaged  in  during  challenging  situations. 
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Because  Study  1  never  obtained  regulatory  approval  to  conduct  human-subjects  testing,  the 
JHU/APL  team  never  finalized  all  the  details  of  this  study,  as  it  was  designed  to  follow  Study  1 . 

4  STUDY  3:  SPECIAL  FORCES  ASSESSMENT  AND 
SELECTION 

4.1  BACKGROUND,  OBJECTIVE,  AND  SCOPE 

Assessment  and  selection  of  elite  military  personnel  remains  an  operational  priority  of  the 
Special  Operations  community  and  an  enduring  research  topic  within  operational  psychology. 
One  obvious  difficulty  is  that  different  people  will  have  different  strengths  and  weaknesses,  and 
in  the  absence  of  knowing  what  is  likely  to  occur  in  the  future,  it  is  impossible  to  determine 
which  particular  strengths  are  most  important.  US  Army  Special  Forces  are  well  aware  of  this. 
Special  Forces  personnel  are  elite  soldiers  who  are  selected  based  not  only  on  physical  criterion 
but  also  personality  attributes  and  cognitive  abilities.  Previous  research  has  found  that  certain 
traits  like  grit  and  physical  ability  (Beal,  2010)  were  significant  predictors  of  selection  for 
Special  Forces.  As  the  world  has  changed,  however,  so  has  the  criterion  for  which  selection  and 
assessment  should  occur.  There  is  an  interest  by  the  Army  to  better  understand  their  current 
selection  and  assessment  process  and  continue  working  to  improve  that  process. 

JHU/APL  worked  with  JFKSWCS  to  analyze  historical  SFAS  data.  This  dataset  included  many 
physical  and  cognitive  traits,  many  relevant  for  cross-cultural  interactions,  such  as  ability  to  learn 
foreign  languages  and  foreign  language  testing  ability.  The  goal  of  this  study  was  to  better 
understand  the  SFAS  selection  process.  Are  they  overweighing  certain  variables  and  ignoring 
others  that  should  be  considered?  Are  there  any  ways  to  streamline  the  course  to  make  it  more 
efficient? 

4.2  METHODS  AND  DATASET 

The  dataset  was  of  all  soldiers  who  attempted  the  SFAS  course  in  FY2013  and  FY2014,  a 
sample  size  of  6,112.  Of  these  soldiers,  approximately  600  were  injured  during  the  course  and 
were  removed  from  subsequent  analyses.  Of  the  remaining  5,519  soldiers,  roughly  half  passed 
the  course  and  were  selected  to  be  in  Special  Forces  (2,605  to  be  exact),  with  some  of  the  non- 
selected  voluntarily  withdrawing,  some  involuntarily  withdrawing,  and  some  involuntarily 
withdrawn  for  failing  physical  tests  during  the  early  part  of  the  course. 

For  each  soldier,  there  were  at  most  42  different  metrics  collected  (42  was  the  maximum 
possible,  but  almost  all  soldiers  were  missing  some  metrics).  These  metrics  were  then  treated  as 
variables  in  statistical  models.  These  included  variables  such  as  their  military  occupational 
specialty  (MOS),  Ranger  qualification,  and  Post  from  which  they  came.  Other  variables  focused 
more  on  demographic  characteristics  including  age,  education  level,  and  time  in  service.  Various 
measures  of  cognitive  ability  were  also  included,  including  Intelligence  Quotient  (IQ  scores, 
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including  full-scale,  performance,  and  verbal),  General  Technical  (GT)  score  (a  measure  of 
crystallized  intelligence  tested  as  part  of  the  Armed  Services  Vocational  Aptitude  Battery  or 
ASVAB),  Defense  Language  Aptitude  Battery  (DLAB)  score,  and  some  measures  of  reading  and 
math  ability.  Numerous  physical  metrics  were  collected  as  well  including  push-ups,  sit-ups,  and 
pull-ups  counts,  run-time  for  a  two-mile  course,  obstacle  course  performance,  and  rucksack 
performance  times.  Finally,  there  were  a  few  variables  that  combined  physical  and  cognitive 
performance.  Starl  and  star2  were  land  navigation  tasks  that  required  both  physical  and 
cognitive  prowess  to  succeed.  Table  3  includes  the  set  of  variables  included  in  the  subsequent 
analyses.  Not  included,  but  which  would  be  worthwhile  to  do  so  in  future  research  are  social 
variables  (Adaptability,  Capability,  Courage,  Integrity,  Perseverance,  Personal  Responsibility, 
Professionalism,  and  Team  Player)  evaluated  by  both  peers  and  Cadre  during  team  events  toward 
the  end  of  SFAS  as  well  as  psychometric  variables  (personality  and  cultural  intelligence). 


Table  3.  SFAS  Variables  Included  in  Analysis 


Variables 

Explanation 

Administrative  Variables 

COMP 

Active  Duty  or  National  Guard  (Officer  &  Enlisted) 

GRADE 

Pay  grade 

MOS 

Military  Occupational  Specialty 

POST 

Base  to  which  they  were  assigned  prior  to  SFAS 

STATE 

Demographic  Variables 

RGR 

Ranger  qualification  (Y  or  N) 

ED 

Years  of  education 

TIS 

Time  in  service 

Mental  Variables 

READ 

Tests  of  Adult  Basic  Education  (TABE)  Reading  skills 

MATH 

TABE  Math  skills 

LANG 

TABE  Language  skills 

GT 

General  Technical  score  (ASVAB) 

DLAB 

Defense  Language  Aptitude  Battery 

DLPT 

Defense  Language  Proficiency  Test 

VIQ 

Verbal  IQ 

PIQ 

Performance  IQ 

FSIQ 

Full  Spectrum  IQ 

ST 

Skilled  Technical  (alternative  language  score) 

Physical  Variables 

SWIM 

Swim  qualification  (not  required) 

PU 

Push-ups  (raw) 

SCORE 

Push-ups  score  (converted) 

su 

Sit-ups  (raw) 
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SCORE 

Sit-ups  score  (converted) 

RUN 

Run  time 

SCORE 

Run  score  (converted) 

TOTAL 

Sum  PU,  SU,  Run 

RUN  1 

Timed  run 

RUCK  1 

Timed  ruck 

RUN  2 

Timed  run 

RUCK  2 

Timed  ruck 

SCORE 

Score  of  timed  events 

O'COURSE 

Obstacle  course 

PULL-UP 

Pull-ups  (raw) 

Tactical  Variables 

MAP  TEST 

Test  of  land  navigation  concepts 

PEI 

Practical  exercise 

PE2 

Practical  exercise 

PE3 

Practical  exercise 

PE4 

Practical  exercise 

TOTAL 

PE  Total  Score 

STAR1 

Graded  land  navigation  exercise 

STAR2 

Graded  land  navigation  exercise 

TOTAL 

Sum  STAR1  &  2 

An  interesting  aspect  of  this  dataset  is  that  the  missing  variables  were  often  informative.  A 
soldier  who  is  missing  lots  of  data  likely  is  missing  that  data  because  they  did  not  participant  in 
the  later  stages  of  the  course,  meaning  they  have  no  chance  of  being  selected.  Thus,  the  analyses 
would  have  to  allow  for  the  fact  that  variables  that  were  collected  later  in  the  course  will  only 
have  data  from  the  subset  of  soldiers  who  succeeded  in  the  course  up  to  that  stage. 

4.3  SELECTION  ANALYSES 

In  order  to  control  for  the  fact  certain  variables  were  missing,  values  for  the  missing  variables 
were  imputed  in  stages.  In  Stage  1,  about  500  soldiers  were  removed  from  training.  The  first 
stage  of  imputation  only  imputed  variables  from  these  500  soldiers  that  they  would  have  done  in 
Stage  2.  Each  variable  was  imputed  sequentially,  so  for  example  Run  1  would  be  imputed  from 
variables  that  were  present  in  all  5,519  participants  with  Run  being  the  most  informative  in 
imputing  Run  1 .  Once  Run  1  was  imputed,  then  a  new  model  would  be  fit  to  fit  another  variable, 
such  as  fsiq,  in  which  case  education  may  be  most  influential  in  imputing  this  variable.  After 
imputing  the  variables  missing  in  Stage  1,  roughly  1,000  soldiers  were  cut  in  Stage  2.  Once 
again,  variables  were  imputed  for  these  soldiers.  The  following  variables  were  imputed  at  this 
stage:  o. course,  map.test,  pel,  pe2,  pe3,  pe4,  starl,  and  star2. 
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For  all  analyses  where  a  model  was  tested,  the  data  was  first  divided  into  training  and  test  sets. 
The  training  set  was  used  to  train  a  model,  and  the  test  set  was  used  to  see  performance  of  the 
model  on  data  where  it  was  not  trained. 

4.3.1  Logistic  Regression  Models 


The  first  test  conducted  was  a  replication  of  Beal  (2010),  who  used  logistic  regression  to  predict 
SFAS  selection.  Beal  did  not  report  all  of  his  results,  but  he  did  report  an  adjusted  R-squared  of 
0.42  from  his  logistic  regression  model.  He  found  a  five  variable  model  worked  best,  using  a  1st 
ruck  march,  TABE  score,  their  ambition  (as  measured  by  a  subscale  from  a  test  of  grit),  their 
years  of  education,  and  their  Army  Physical  Fitness  Test  (APFT)  score. 

To  replicate  his  findings,  the  variables  were  standardized  via  a  Z-transform.  Next,  forward 
variable  selection  was  used  to  determine  which  variables  were  most  significant.  This  resulted  in 
the  following  model: 


##  Call: 

##  glm(formula  =  wasselected  ~  starl  +  ruck. 2  +  star2  +  fsiq  +  ruck.l  + 
##  pu  +  dlab  +  rgr  +  pel  +  dist  +  o. course  +  lang  +  ed  +  map. test  + 

##  run.l  +  pull-up^  family  =  "binomial",  data  =  train) 

## 

##  Deviance  Residuals: 

##  Min  IQ  Median  3Q  Max 

##  -3.0564  -0.6465  -0.2447  0.7026  3.7611 

## 


##  Coefficients: 

##  Estimate  Std.  Error  z  value  Pr ( > | z | ) 

##  (Intercept)  4.356e-01  5.696e-02  7.647  2.05e-14  *** 


##  starl 
##  ruck. 2 
##  star2 
##  fsiq 
##  ruck.l 
##  pu 
##  dlab 


8 . 863e-01  5 . 896e-02  15.033  <  2e-16  *** 
-4.476e-01  7 . 750e-02  -5.776  7.66e-09  *** 
7 . 788e-01  5 . 964e-02  13.058  <  2e-16  *** 

3 . 281e-01  5 . 778e-02  5.678  1.37e-08  *** 

-4 . 356e-01  8 . 028e-02  -5.426  5.78e-08  *** 
2.002e-01  5 . 190e-02  3.857  0.000115  *** 

-3 . 570e-01  9 . 832e-02  -3.631  0.000282  *** 


##  rgr  -5 . 921e-01  1.521e-01  -3.892  9.94e-05  *** 
##  pel  1 . 364e-01  5.221e-02  2.613  0.008966  ** 

##  dist  -5 . 692e-08  1.985e-08  -2.867  0.004137  ** 
##  o. course  1.314e-01  5.398e-02  2.434  0.014923  * 

##  lang  2 . 483e-01  9.243e-02  2.686  0.007225  ** 

##  ed  -1 . 124e-01  5.172e-02  -2.173  0.029814  * 

##  map. test  1.050e-01  5.585e-02  1.880  0.060069  . 

##  run.l  1 . 040e-01  5.432e-02  1.914  0.055592  . 

##  pull. up  8 . 941e-02  5.186e-02  1.724  0.084712  . 

##  --- 


##  Signif.  codes:  0  '***'  0.001  '**'  0.01  '*'  0.05  0.1 
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## 

##  (Dispersion  parameter  for  binomial  family  taken  to  be  1) 
## 

##  Null  deviance:  5118.7  on  3699  degrees  of  freedom 
##  Residual  deviance:  3281.6  on  3683  degrees  of  freedom 
##  AIC :  3315.6 


The  Nagelkerke  R-squared  for  this  model  was  0.52,  greater  than  Beal’s,  but  a  very  different 
model.  The  most  significant  predictor  was  starl,  the  land  navigation  task,  followed  by  the  second 
ruck  sack  test  and  then  star2,  the  second  land  navigation  task.  Ultimately,  this  model  used  18 
variables.  To  make  a  more  meaningful  comparison  with  Beal’s  result,  a  logistic  regression  model 
was  created  with  only  the  top  five  variables  (starl,  ruck.2,  star2,  PIQ,  and  ruck.l).  The 
Nagelkerke  R-squared  for  this  model  was  0.49,  larger  than  from  the  model  in  Beal’s  paper. 

A  more  important  test,  however,  is  how  accurately  this  model  predicts  selection  on  a  test  dataset. 
Results  are  below.  The  table  shows  the  actual  results.  The  top  row  is  all  the  cases  where  the 
model  predicts  an  individual  will  not  be  selected,  and  the  bottom  row  shows  the  cases  where  the 
model  predicts  an  individual  will  be  selected.  When  the  model  predicts  the  participant  will  not  be 
selected,  the  model  makes  an  erroneous  prediction  (a  false  negative)  192  times,  where  the  soldier 
was  actually  selected  despite  the  model’s  prediction.  The  model  generates  a  false  positive  170 
times,  where  it  predicts  selection  and  the  individual  is  not  selected. 

computeClassificationStats(tbl) 

##  Actual 
##  Pred  0  1 

##  0  745  192 

##  1  170  712 

##  $Accuracy 
##  [1]  0.8009896 
## 

##  $Precision 
##  [1]  0.8072562 
## 

##  $Recall 

##  [1]  0.7876106 

## 

##  $FScore 
##  [1]  0.7973124 

The  model  is  pretty  accurate,  with  an  accuracy  rate  of  80.1%.  Also  reported  are  precision,  the 
number  of  correct  predictions  of  selection  over  all  predictions  of  selection,  here  712/(170+712), 
and  recall,  the  number  of  correct  predictions  of  selection  divided  by  all  the  actual  cases  of 
selection,  here  712/(712+192).  Finally,  the  harmonic  mean  of  recall  and  precision,  the  F-score,  is 
reported  and  it  has  a  score  of  79.7%. 
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4.3.2  Recursive  Partitioning  and  Regression  Tree  Model 

A  logistic  regression  model,  although  accurate,  can  be  difficult  to  interpret.  So  other  models 
were  tested  that  ultimately  performed  even  more  accurately  and  were  easier  to  interpret.  One  of 
these  models  used  a  recursive  partitioning  and  regression  tree  (RPART),  and  resulted  in  the 
following  model  that  only  used  two  variables,  starl  and  star2,  the  two  land-navigation  tasks: 


star2  <  0.11 


Rattle  2015-Sep-02  16:05:12  Jonathon 


Figure  5:  Tree  to  predict  SFAS  Outcome 


To  interpret  this,  the  0  or  1  in  the  top  of  each  box  is  the  prediction  from  the  model.  If  the  model 
predicts  that  all  participants  will  not  be  selected  (the  0),  then  the  model  will  have  an  accuracy  of 
53%.  However,  if  starl  for  an  individual  is  greater  than  0.01  standard  deviations  above  the  mean, 
i.e.  above-average,  then  the  model  predicts  that  individual  will  be  selected  (the  1)  and  the  model 
will  have  an  accuracy  of  77%  on  those  individuals  who  scored  above-average  on  starl. 


If  the  participant  performed  at  average  level  or  below  on  starl,  then  the  model  also  utilizes 
performance  at  star2.  If  star2  was  more  than  0.11  standard  deviations  above  the  mean,  then  the 
model  predicts  the  participant  would  be  selected.  Otherwise,  the  model  predicts  the  participant 
would  not  be  selected. 


The  performance  of  the  model  on  the  test  set  is  below: 

computeClassif icationStats(tbl) 
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##  Actual 
##  Pred  0  1 

##  0  638  54 

##  1  278  849 

##  $Accunacy 
##  [1]  0.8174821 
## 

##  $Precision 
##  [1]  0.7533274 
## 

##  $Recall 

##  [1]  0.9401993 

## 

##  $FScore 
##  [1]  0.8364532 

With  this  model,  it  very  rarely  generates  a  false  negative;  only  54  times  does  it  predict  a  person 
would  not  be  selected  who  would  have  been.  Compare  this  to  278  times  where  it  makes  a  false 
positive  prediction.  This  manifests  itself  with  a  very  high  recall  of  94%,  but  a  lower  precision 
score  of  only  75%.  However,  the  F-Score  and  the  area-under-the-curve  (Figure  6)  both  show  the 
strong  predictive  power  of  this  model. 


Figure  6:  AUC  for  RPART  Tree 
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This  model,  using  only  two  variables,  has  higher  accuracy  and  F-scores  than  the  more 
complicated  logistic  regression  using  18  variables.  This  indicates  that  just  focusing  performance 
on  those  two  variables  was  sufficient  to  make  meaningful  predictions.  In  fact,  this  two-variable 
model  outperfonns  logistic  regression  using  all  variables,  which  ovcrlits  in  the  training  dataset  so 
has  an  F-score  of  only  78.3%  in  the  test  dataset. 

The  false  negatives  and  false  positives  from  this  analysis  were  often  caused  by  people  who  got 
outstanding,  or  poor,  reviews  from  their  instructors  and  classmates,  according  to  discussions  with 
personnel  at  Fort  Bragg  and  Camp  Mackall.  High  performance  on  these  tasks  was  not  enough  to 
overcome  a  reputation  for  causing  trouble  or  not  working  well  with  the  team. 

4.3.3  Random  Forest  Model 

A  random  forest  model  was  also  generated.  In  a  random  forest,  only  a  subset  of  variables  are 
used  at  any  time  for  any  given  tree,  and  then  numerous  iterations  are  run  to  see  which  trees  work 
best  where  different  iterations  often  use  only  different  subsets  of  variables.  A  random  forest  will 
always  outperform  an  RPART,  because  it  chooses  the  best  trees  amongst  numerous  iterations. 
The  downside  of  a  random  forest  is  there  can  be  no  average  tree,  because  each  iteration  contains 
different  variables  and  thus  are  different.  The  random  forest  model  was  used  to  evaluate  how 
well  the  RPART  model  performed  compared  to  more  complicated  tree  models. 

The  key  results  from  the  random  forest  model  can  be  seen  in  Figure  7.  With  random  forests, 
there  are  two  main  ways  to  measure  the  importance  of  the  variables.  The  left  plot  of  Figure  7 
shows  the  variable  importance  plot,  which  measures  how  much  worse  the  model  performs  when 
a  variable  is  randomly  permuted.  The  larger  the  mean  decrease  in  accuracy,  the  more  important 
that  variable  is  to  making  accurate  predictions.  In  this  case,  starl,  ruck.2  and  star2  all  play  a 
particularly  integral  role  in  accurate  prediction  of  SFAS  performance.  Another  metric,  shown  in 
the  right  plot,  is  to  measure  the  importance  via  the  Gini  impurity  index.  Gini  impurity  is  a 
measure  of  how  variable  or  impure  the  final  predicted  classes  in  a  model  are.  The  more  the  data 
of  the  final  predicted  class  only  has  members  of  that  class,  the  lower  the  Gini  impurity  value.  A 
larger  decrease  in  Gini  means  that  variable  is  more  important  in  separating  the  predicted  classes, 
i.e.  without  that  variable  there  is  worse  separation  of  the  final  classes.  In  terms  of  decreasing 
Gini,  starl  and  star2  have  considerably  larger  effects  than  do  any  of  the  other  variables.  Thus 
between  the  two  plots,  starl  and  star2  are  the  only  variables  that  seem  to  be  important  regardless 
of  what  metric  is  used.  The  fact  that  the  second  rucksack  test  goes  from  being  one  of  the  most 
important  variables  by  one  metric  to  a  distant  fourth  indicates  that  it  is  not  nearly  as  predictive  as 
starl  and  star2. 
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Figure  7:  Random  Forest  Summary  of  results.  Points  more  to  the  right  indicate  the  model  does  not  perform 
as  well  when  that  variable  is  removed,  either  via  a  decrease  in  accurate  predictions  or  a  decrease  in  how  well 
separated  the  final  predictions  are.  Starl  and  star2  are  the  two  variables  that  are  most  important  across  both 

of  these  measures. 

The  random  forest  model  performance  on  the  test  dataset  was: 

computeClassif icationStats(tbl .forest) 

##  Actual 
##  Pred  0  1 

##  0  688  88 
##  1  228  815 

##  $Accunacy 
##  [1]  0.8262782 
## 

##  $Precision 
##  [1]  0.7813998 
## 

##  $Recall 

##  [1]  0.9025471 

## 

##  $FScone 
##  [1]  0.8376156 
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Performance  was  better  than  the  RPART  model,  but  not  significantly  more  so  once  the 
complexity  of  the  model  is  considered.  Accuracy  with  the  random  forest  model  was  82.6% 
compared  to  81.7%  with  the  RPART  model,  but  the  F-score  for  the  random  forest  model  was 
83.76%,  only  slightly  better  than  the  83.64%  F-score  from  the  RPART  model.  This  indicates  that 
the  RPART  model  is  quite  a  good  fit  and  that  no  real  need  to  add  the  complexity  of  a  random 
forest. 

4.4  ANALYZING  SFAS  ATTRIBUTES  USING  NORMALIZED  MUTUAL 
INFORMATION  (NMI) 

The  goals  discussed  below  for  analyzing  the  SFAS  data  set  include:  (1)  understanding 
relationships  between  the  data  attributes,  and  in  particular  the  correspondence  between  the 
candidates’  characteristics  (background  and  various  test  scores)  and  the  selection  outcome;  (2) 
building  a  classifier  to  predict  which  candidates  are  likely  to  be  selected. 

4.4.1  Background  and  Motivation  for  the  NMI  Approach 

The  SFAS  data  consists  of  a  mix  of  categorical  and  numerical  variables.  Categorical  variables 
are  those  for  which  there  is  no  natural  arithmetic  (such  as  sums,  averages,  etc.).  Some  of  the 
categorical  variables  included  in  SFAS  data  are  Comp,  Grade,  MOS,  Post,  State,  Rgr,  and  Status. 
When  a  data  set  contains  categorical  variables,  methods  that  measure  dependency  between 
numerical  variables,  such  as  correlation,  cannot  be  applied. 

An  additional  challenge  in  analyzing  SFAS  data  is  the  relatively  high  number  of  variables;  this  is 
an  instance  of  the  “curse  of  dimensionality.”  The  curse  of  dimensionality  is  the  principle  that,  as 
the  number  of  variables  (or  dimensions)  increases,  the  observed  data  necessarily  becomes  sparse 
within  the  total  space  of  possible  value  combinations.  For  instance,  If  there  are  just  10 
categorical  variables  and  each  variable  can  assume  only  4  possible  values,  the  number  of 
possible  observed  data  values  (i.e.  value  combinations)  is  4A 10,  which  is  over  1  million.  In  most 
situations,  including  the  SFAS  data,  the  set  of  observed  data  is  much  smaller.  Most  value 
combinations  are  therefore  never  observed,  and  those  that  are  observed  are  observed  usually  only 
once.  The  consequence  of  data  sparsity  is  that  it  is  difficult  to  derive  informative  probability 
distributions,  and  therefore  also  difficult  to  derive  anomaly  detection  methods,  classifiers,  etc. 
which  rely  on  probabilities. 

Therefore,  two  significant  challenges  in  analyzing  the  SFAS  data  are:  (1)  which  metrics  to  apply, 
since  correlation-type  metrics  for  numerical  data  are  unusable,  and  (2)  how  to  overcome  data 
sparsity. 

The  proposed  approach  developed  at  JHU/APL  uses  mutual  information  to  characterize  variable 
dependencies.  The  variable  dependencies  can  then  be  used  to  derive  a  probability  distribution  for 
the  data,  and  in  turn  the  probability  distribution  can  be  used  to  build  a  classifier. 
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4.4.2  Definition  of  NMI 


Normalized  mutual  information  (NMI)  measures  dependency  between  pairs  of  categorical 
variables.  Pairwise  NMI  values  range  between  [0,1],  where  0  means  the  variables  are 
independent,  and  1  indicates  complete  dependence.  The  NMI  on  variables  X  and  Y  is  given  by 
M(X,Y)  in  this  definition: 


7(Y,7) 


M(x.r>=- 

nmi(H(X).H(r)) 

In  the  above  formula,  I(X,Y)  is  the  mutual  information  (non-normalized),  and  H  is  variable 
entropy: 


I(X.Y)=££p(x,v)1°: 

xeXyeY 


p(x.y)  ) 
p(x)p(y)j 


H(X)  =  ~YJP(x)  logO(.v)) 

xeX 


4.4.3  Binning  Numerical  and  Categorical  Data 

NMI  must  be  applied  to  categorical  data.  This  means  that  any  numerical  data  (such  as  ages, 
various  types  of  test  scores,  etc.)  must  first  be  binned.  The  original  values  can  sometimes  be  used 
as  the  bins.  For  instance,  in  the  SFAS  data,  consider  the  observed  age  values  such  as  21,  22,  23, 
etc.  all  as  separate  bins.  However,  when  there  are  many  distinct  observed  values,  it  is  often  better 
to  simplify  the  values  into  coarser  bins.  The  choice  of  binning  is  often  informed  by  domain 
knowledge.  For  age,  age  ranges  such  as  20-24,  25-29,  etc.  tend  to  be  used  and  obviously  various 
binning  schemes  can  be  tried  on  each  numerical  attribute. 

Even  with  categorical  data,  the  data  values  are  sometimes  binned  when  the  observed  values 
appear  over-specific.  In  the  SFAS  data,  certain  values  were  combined  together  when  it  made 
sense  to  do  so.  Details  about  that  are  in  the  sections  below. 

The  takeaway  for  both  categorical  data  and  numerical  data  is  that  binning  is  important  and 
affects  the  NMI  results. 


4.4.4  Steps  for  Analyzing  NMI  on  SFAS  Data 

The  following  steps  were  used  to  compute  initial  NMI  results  on  the  SFAS  data. 
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4.4.4.1  Filtering  Records 

Because  there  were  so  many  categories,  initial  analyses  focused  on  active-duty  enlisted 
personnel,  as  they  were  of  particular  interest.  Hence,  before  analyzing  NMI,  some  of  the  data 
records  were  removed  from  the  analysis: 

•  Officer  records  were  eliminated  (indicated  by  Comp  values  of  AGO,  NGO,  or  NG18X) 

•  Medical  withdrawals  were  eliminated  (indicated  by  a  Status  value  of  Medical  Drop) 

•  In  the  binning  process  below,  a  small  number  of  records  were  discarded  due  to  data  with 
unexpected  formats  or  very  odd  values,  likely  typos. 

4.4.4.2  Attributes  and  Binning 

Thus  far,  the  following  attributes  have  been  incorporated  into  the  NMI  analysis,  with  the  binning 
and/or  other  data  value  modifications  as  indicated. 


Table  4.  SFAS  Variables  included  in  NMI  Analysis 


Attribute 

Binning/Modifications 

Notes 

Status 

Original  value,  no  binning 

StatusBinned 

Status  of  selected  is  labeled  as 
“pass”  and  all  other  values  are 
binned  to  a  single  value  of 
“fail” 

Simplifying  the  Status  values  to  binary. 
This  is  the  status  description  desired  if 
the  eventual  goal  is  to  create  a  selected 
vs.  non-selected  classifier. 

StarMax 

Contains  the  maximum  value  of 
starl  and  star2  scores. 

Used  maximum  rather  than  either  the 
individual  score  or  the  sum  of  the 
scores,  since  some  records  have  one  of 
the  scores  missing.  A  missing  value  is 
not  necessarily  indicative  of  a  “bad” 
candidate,  and  therefore  it  was  not 
desirable  to  create  a  simple  “missing” 
placeholder,  or  to  assign  a  value  of  0  to 
missing  data. 

GradeBinned 

Grade  is  binned  into  categories 
of  (1)  E2  &  E3,  (2)  E4  &  E5, 
and  (3)  E6  and  higher 

Any  other  observed  values  caused  the 
record  to  be  omitted  from  the  NMI 
computation. 

State 

Original  value,  no  binning 

Post 

Original  value,  no  binning 

Comp 

Original  value,  no  binning. 

AgeBinned 

Age  is  binned  by  5  year 
increments:  <20,  20-24,  25-29, 
etc. 

As  with  any  numerical  data,  alternate 
histogram  bins  could  be  tried  here. 

Rgr 

Original  value,  no  binning 

MosNum 

The  letter  part  of  the  MOS  data 

Binning  method  suggested  from  earlier 
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value  is  dropped.  For  example, 
“14T”  becomes  “14”. 

discussion. 

EdBinned 

Education  is  binned  as  (1)  high 
school,  corresponding  to  values 
12  and  GED;  (2)  some  college, 
from  values  13-15;  (3)  college 
or  grad  school,  from  values  16 
and  higher;  or  (4)  a  missing 
value. 

Alternative  binning  could  be  tried  here, 
for  instance  by  distinguishing  college 
vs.  grad  school  education  levels. 

RunMax 

Equal  to  the  maximum  of  the 
Runl  and  Run2  scores,  but  no 
other  binning. 

Same  reasoning  as  StarMax  as  to  why 
maximum  value  is  used.  Binning  could 
also  be  incorporated  in  the  future  to 
coarsen  the  score  results. 

RuckMax 

Equal  to  the  maximum  of  the 
Ruckl  and  Ruck2  scores,  but  no 
other  binning. 

Same  reasoning  as  StarMax  as  to  why 
maximum  value  is  used.  Binning  could 
also  be  incorporated  in  the  future  to 
coarsen  the  score  results. 

Using  the  attributes  listed  above,  the  NMI  values  are  computed  for  all  78  pairwise  combinations. 
The  resulting  NMI  scores  are  shown  in  the  graph  in  Figure  8.  The  NMI  is  recorded  on  each  link, 
and  note  the  thicker  links  correspond  to  higher  NMI. 

Some  remarks  about  results: 

•  Since  predicting  outcomes  of  selected  vs.  non-selected  (i.e.  a  two-class  problem)  is  the 
ultimate  goal  of  this  analysis,  the  NMIs  against  the  StatusBinned  node  are  more  relevant 
than  those  against  Status. 

•  StarMax  had  a  NMI  of  0.48  with  StatusBinned,  which  is  a  strong  dependency,  as 
expected. 

•  The  RuckMax  and  RunMax  scores  also  showed  some  dependency  with  Status,  but  the 
dependency  decreased  after  Status  was  coarsened  to  StatusBinned. 

o  Binning  the  numerical  values  on  the  Run/Ruck  scores  can  be  incorporated  to  see 
if  the  dependency  with  StatusBinned  improves.  As  of  now,  these  are  raw 
Run/Ruck  scores:  with  no  binning  being  used  on  the  resulting  numerical  scores. 
This  omission  was  due  only  to  time  constraints  in  generating  the  initial  results, 
o  Note  that  RuckMax  and  RunMax  are  somewhat  related  to  each  other,  as  expected. 

•  The  MOS  and  Grade  attributes  were  not  significantly  related  to  Status  or  StatusBinned. 
This  was  somewhat  unexpected,  according  to  earlier  discussions,  as  it  was  thought  that 
those  with  a  Special  Forces  MOS  would  be  more  likely  to  be  selected  in  SFAS.  It  is 
possible  that  the  binning  schemes  used  in  MOS  and  in  Grade  were  suboptimal. 
Alternative  binning  can  be  tried  on  each  of  those  attributes. 
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4.4.4.3  Additional  Attributes 

Other  attributes  such  as  maptest,  the  various  IQ  tests,  and  other  test  scores  can  be  incorporated  in 
the  future.  This  is  a  somewhat  slow  process  because  test  scores  can  be  complicated  to  handle: 
there  are  many  possible  choices  for  binning  schemes,  as  well  as  the  confounding  issue  of  how  to 
deal  with  missing  data.  In  general,  missing  data  cannot  be  arbitrarily  set  to  0  or  to  a  placeholder 
“missing”  value  without  negatively  impacting  the  NMI.  Because  of  these  complications,  there  is 
a  need  to  incorporate  attributes  carefully  and  consider  various  binning  schemes  for  each  of  them. 
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Figure  8.  Normalized  Mutual  Information  between  all  pairs  of  attributes  contained  the  table.  The  pairwise  NMI 
is  indicated  on  each  link.  Thicker  links  correspond  to  higher  NMI.  The  two  status  attributes  are  highlighted  in 

red  for  emphasis. 
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4.4.4.4  NMI-Based  Classification 

The  NMI  results  can  be  used  to  construct  a  classifier  which  predicts  “Selected”  vs.  “Non- 
Selected”  status  from  a  useful  subset  of  the  non-Status  attributes.  The  classifier  has  been  applied 
successfully  to  other  datasets,  such  as  a  publicly  available  mushroom  data  set.  In  that  application, 
the  classifier  was  used  to  predict  poisonous  vs.  non-poisonous  mushrooms,  and  performed  well 
against  test  data. 

Because  of  the  binning  issues  and  missing  data  issues  which  are  still  to  be  resolved, 
implementing  a  final  classifier  for  the  SFAS  data  is  not  achievable  in  the  current  period  of 
performance.  However,  a  few  general  principles  can  be  noted  here.  The  classifier  will  work 
better  if  (1)  dependencies  of  the  non-Status  attributes  with  StatusBinned  are  high;  (2) 
dependencies  of  the  non-Status  attributes  with  each  other  are  low.  Intuitively,  this  is  because  if 
the  non-Status  attributes  are  relatively  independent,  they  each  provide  a  useful  piece  of 
information  towards  predicting  the  StatusBinned  outcome.  By  contrast,  if  these  attributes  are 
very  mutually  dependent  (effectively,  more  redundant),  then  they  each  contribute  less  new 
information  into  the  classifier.  Therefore,  it  is  ideal  to  bin  attributes  in  a  way  that  achieves  NMI 
behavior  described  in  (1)  and  (2). 

4.5  DISCUSSION 

SFAS  selection  was  well-summarized  by  just  the  two  land-navigation  tasks,  starl  and  star2.  Just 
from  knowing  performance  on  these  two  variables,  it  is  possible  to  accurately  predict  selection 
83%  of  the  time.  That  just  these  two  variables,  representing  one  skill,  are  so  determinative  of 
selection  has  a  few  important  implications  that  cannot  be  resolved  as  yet.  One  is  that  perhaps 
SFAS  training  time  could  be  reduced  if  these  tasks  were  moved  to  earlier  in  training. 
Unfortunately,  the  data  itself  cannot  resolve  this,  because  the  fact  that  the  land  navigation  occurs 
late  in  training  may  itself  be  important.  Perhaps  those  who  have  passed  the  grueling  tasks  earlier 
in  training  and  now  have  to  go  through  the  land  navigation  course  and  overcome  their  fatigue  are 
those  who  should  be  selected.  If  the  land  navigation  task  occurred  earlier  in  training,  the  soldiers 
would  be  fresher  and  thus  have  less  need  to  overcome  fatigue  and  the  grind  of  the  course  to  both 
plan  and  do  a  physical  task  while  exhausted,  which  may  make  an  early  land  navigation  task  less 
diagnostic  of  future  performance.  Second,  it  is  possible  that  some  of  the  tasks  could  be  removed 
from  training  because  they  were  not  diagnostic  of  selection.  Again,  the  data  cannot 
unambiguously  assert  this.  If  overcoming  fatigue  is  a  critical  part  of  the  land  navigation  task, 
then  earlier  tasks  that  contributed  to  fatigue  matter  indirectly,  even  if  they  have  no  direct  effect. 

An  even  more  critical  caveat  is  that  the  goal  of  SFAS  is  to  select  those  soldiers  who  will  perform 
well  not  only  in  the  SFQC  but  also  in  the  Special  Forces  Regiment.  Currently,  the  data  on  which 
soldiers  perform  best  after  the  SFAS  course  completes  is  somewhat  deficient.  To  rectify  this  in 
part,  JHU/APL  is  working  with  1st  SWTG(A)  to  obtain  SFQC  data  to  determine  how  specific 
SFAS  variables  predict  SFQC  performance. 
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The  ideal  data,  however,  is  performance  data  once  the  SF  Sergeant  or  Officer  graduates  SFQC 
and  joins  the  Special  Forces  Regiment.  There  is  currently  no  clear  objective  measure  of 
performance  outside  the  Total  Army  Performance  Emulation  System,  and  that  data  is  not 
available  to  researchers.  If  there  was  some  measure  of  which  SF  perform  better,  than  it  would  be 
possible  to  determine  if  SFAS  was  selecting  the  correct  personnel  or  not.  Absent  that,  it  is 
possible  to  determine  which  tasks  are  most  critical  to  selection  as  it  is  currently  conducted,  in 
order  to  make  decision  makers  more  informed  that  this  is  what  determines  the  current  outcomes, 
so  they  can  decide  if  those  tasks  are  what  they  consider  most  critical  for  future  Special  Forces 
operators. 


5  CONCLUSION 


Research  designs  to  identify  the  underlying  cognitive  aspects  (and  their  neural  correlates)  with 
high  ecological  validity  of  relevance  for  the  military  are  underrepresented  in  the  academic 
literature,  and  therefore  research  designs  such  as  those  proposed  for  the  NC3A  project  are 
needed.  There  was,  is,  and  will  be  a  need  to  better  understand  some  of  the  “soft”  skills  that 
military  personnel  need  to  succeed  in  across  the  range  of  military  operations.  Cross-cultural 
interactions  are  likely  to  be  a  key  skill  needed  for  success  in  the  near-future,  and  it  seems 
unlikely  that  military  personnel  will  be  able  to  determine  which  areas  or  cultures  in  particular 
will  be  the  next  flashpoint,  given  the  number  of  potential  flashpoints  in  the  world.  Despite  the 
regional  alignments  of  many  major  units  in  the  Army  and  Marine  Corps,  the  linguistic  and 
cultural  diversity  within  those  areas  of  operation  will  necessitate  culture-general  skills. 

The  problem,  of  course,  is  “soft”  skills  are  more  nebulous  and  hard  to  objectively  measure.  This 
project  was  meant  to  help  that  along;  although  due  to  regulatory  hurdles,  no  data  that  would  shed 
light  on  this  was  produced.  Human  subject  research  is  not  without  bureaucratic  challenges  but 
the  benefit  of  persevering  through  such  issues  is  external  validity  and  operational  utility.  The 
pertinent  psychological  phenomena  need  to  be  studied  in  the  personnel  whom  the  issues  are  most 
likely  to  manifest  and  in  the  context  in  which  they’re  most  likely  to  operate.  This  simply  cannot 
be  accomplished  using  traditional  undergraduate  research  participants  (Henrich,  Heine,  & 
Norenzayan,  2010). 

A  component  of  this  project  did  examine  historical  SFAS  data  from  a  group  (US  Army  Special 
Forces),  who  are  well  aware  of  the  need  for  cross-cultural  adaptation  and  know  that  their 
operators  need  to  improve  in  this  regard.  This  project  examined  how  SF  makes  this  selection 
right  now,  and  discovered  that  there  was  a  clear  focus  on  land  navigation  skills,  and  not  much 
focus  on  cross-cultural  skills  that  could  be  relevant  for  future  success.  The  receptiveness  of  not 
only  supporting  human  subjects  research  but  also  the  idea  of  cross-cultural  adaptability  of 
JFKSWCS  is  promising  and  provides  some  degree  of  validation  of  the  proposed  designs 
described  earlier.  Unfortunately,  the  lack  of  empirical  data  to  answer  the  aforementioned 
research  questions  limits  where  that  conversation  can  go. 
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5.1  FUTURE  DIRECTIONS 

A  positive  result  of  ONR’s  decision  not  to  proceed  with  HRPO  approval  was  the  opportunity  to 
analyze  historic  SFAS  data.  The  effort  has  been  beneficial  in  ongoing  internally-funded 
JHU/APL  research  to  determine  whether  electroencephalography  (EEG)  and  psychophysical 
(RT)  responses  during  cognitive  control  tasks  improve  an  SFAS  prediction  model.  The  logical 
succession  is  to  obtain  SFQC  data  to  extend  the  prediction  model  and  potentially  incorporate 
neural  correlates  in  said  model. 

Given  the  aforementioned  receptiveness  of  JFKSWCS  to  JHU/APL  research,  rethinking  Study  1 
with  an  SF  cohort  may  be  tractable.  Conceivably  the  snowball  sampling  discussed  above  in 
Study  1  could  be  incorporated  with  SF,  where  participants  asked  to  nominate  other  SF  soldiers 
they  consider  particularly  able.  The  incorporation  of  training/evaluation  observations  from  the 
SFQC  Phase  1  cross  cultural  communication  module  may  not  only  inform  future  task  design  but 
also  provide  an  outcome  measure  for  which  the  assessment  described  in  Study  1  could  serve  as 
predictors. 

The  NC3A  project  was  a  missed  opportunity  for  both  ONR  (to  support)  and  JHU/APL  (to 
execute)  scientifically  rigorous,  operationally  relevant  psychological  research.  In  general,  there  is 
still  a  need  to  have  expertise  classified  so  that  skills  and  traits  that  separate  expert  performers 
from  non-experts  can  be  better  discerned.  Once  those  skills  are  known,  the  best-abled  soldiers 
can  be  selected  and  trained,  at  least  in  theory. 

Military  psychology  has  historically  bridged  the  gap  between  relevant  psychological  research 
and  preparation  of  personnel  ((Windle  &  Vallance,  1964)  and  should  continue  to  do  so.  The 
human  factors  of  warfare  make  psychological  science  particularly  important  to  not  only 
understanding  the  phenomena  but  also  addressing  the  consequences.  The  social  and  behavioral 
sciences  have  much  to  contribute  to  military  doctrine,  organization,  material,  logistics,  personnel, 
and  facilities  (or  DOTMLPF)  (Scales,  2009).  From  identifying  the  underlying  relevant 
psychological  phenomena  through  primary  research  to  advising  on  how  best  to  incorporate  said 
research  into  policy  (Kennedy,  C.  H.,  &  Williams,  2011)  or  training  individuals  in  ways  that 
reflect  said  findings  to  improve  tactical  effectiveness  (Murray,  2013),  military  psychology 
should  play  a  prominent  role  provided  it  has  the  opportunity  to  do  so. 


36 


Research  and  Exploratory 


Development  Department 

REDD-2015-384 

APPENDIX  A:  ABBREVIATIONS 


APFT 

Army  Physical  Fitness  Test 

ARL 

Army  Research  Lab 

CAOCL 

Center  for  Advanced  Operational  Culture  Learning 

CQS 

Cultural  Intelligence  Scale 

DLAB 

Defense  Language  Aptitude  Battery 

DLPT 

Defense  Language  Proficiency  Test 

DoD 

Department  of  Defense 

ED 

Years  of  education 

EIS 

Emotional  Intelligence  Scale 

ERQ 

Emotion  Regulation  Questionnaire 

fMRI 

Functional  Magnetic  Resonance  Imaging 

FSIQ 

Full  Spectrum  IQ 

GT 

General  Technical  score  (ASVAB) 

GT 

General  Technical  Score 

HRPO 

Human  Resources  Protection  Officer 

IRB 

Institutional  Review  Board 

JFKSWCS 

John  F.  Kennedy  Special  Warfare  Center  and  School 

MOS 

Military  Occupational  Specialty 

MPQ-40 

Multicultural  Questionnaire  Short-Form 

NC3A 

Neural  Correlates  of  Cross-Cultural  Adaptation 

NFC 

Need  for  Cognition  Scale 

NMI 

Normalized  Mutual  Information 

ONR 

Office  of  Naval  Research 

PIQ 

Performance  IQ 

RF 

Random  Forest 

RGR 

Ranger  qualification  (Y  or  N) 

RMES 

Reading  the  Mind  in  the  Eyes 

RPART 

Recursive  Partitioning  and  Regression  Tree 

SF 

Special  Forces 

SFAS 

Special  Forces  Assessment  and  Selection 

SFQC 

Special  Forces  Qualification  Course 

TIS 

Time  in  service 

USMC 

United  States  Marine  Corps 

VIQ 

Verbal  IQ 

WM 

Working  Memory 
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APPENDIX  B:  CULTURAL  SENSEMAKING  VIGNETTES 


The  following  are  some  of  the  vignettes  prepared  as  part  of  the  Cultural  Sensemaking  task: 

DIALOGUES 

A.1  01CDP1:  PERFORMANCE  EVALUATION  (STORTI32) 

Word  Count:  140 

Intro:  You  are  assigned  as  the  manager  of  a  branch  of  a  U.S.  company  in  the  Middle  East.  After 
3  months  you  decide  it ’s  time  to  give  performance  evaluations  to  the  local  staff  Khalil  is  the  first 
staff  member  on  your  list. 

YOU:  Thanks  for  coming,  Khalil.  Let’s  go  over  this  evaluation  together,  shall  we. 

KHALIL :  Whatever  you’d  like . 

YOU:  As  you  know,  you’re  quite  strong  in  most  areas.  There  are  just  a  couple  of  areas 

where  you  could  be  stronger. 

KHALIL:  I  see. 

YOU:  One  is  in  writing,  which  isn’t  easy  for  you,  is  it? 

KHALIL:  No. 

YOU:  And  the  other  is  in  identifying  training  needs.  Some  of  your  staff  could  use  more 

computer  training  in  particular. 

KHALIL:  Yes. 

YOU:  Anyway,  it’s  all  written  here.  You  can  read  it.  Otherwise,  no  serious  problems. 

KHALIL:  I’m  very  sorry  to  disappoint  you. 


A.2  02CDP1 :  EXPLANATIONS  (STORTI38) 

Word  Count:  101 

Intro:  You  are  overseeing  production  at  a  U.S.  textile  company  in  China.  One  day,  Miss  Chung, 
a  local  seamstress  comes  to  your  office. 


YOU: 

MISS  CHUNG: 

Miss.  Chung.  What  can  I  do  for  you? 

Excuse  me.  I  need  some  help  with  this  new  machine. 

YOU: 

MISS  CHUNG: 

Of  course.  Let  me  explain  it  again. 

I  asked  Li,  but  she  couldn’t  help  me. 

YOU: 

No,  she  hasn’t  tried  it  yet. 
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MISS  CHUNG:  It’s  a  little  bit  complicated. 

YOU:  It’s  very  complicated,  but  after  I  explained  it  to  you  and  asked  you  if  you 

understood,  you  said  yes. 

MISS  CHUNG:  Yes.  Please  excuse  me. 


A.3  03CDP1 :  WRITING  A  REPORT  (STORTI40) 

Word  Count:  115 

Intro:  You  are  the  manager  for  a  branch  of  a  U.S.  technology  firm  in  India.  You  have  asked  Ram, 
a  local  team  leader  to  evaluate  production  at  the  company’s  five  production  facilities  in  India 
and  to  submit  a  report  outlining  the  results. 

YOU:  How  is  the  evaluation  going,  Ram? 

RAM:  It’s  finished.  We  can  start  on  the  report  anytime  now. 

YOU:  Good.  How  long  do  you  think  it  will  take? 

RAM:  Pardon? 

YOU:  To  write  the  report. 

RAM:  I  couldn’t  say. 

YOU:  You  don’t  know  how  long  it  will  take? 

RAM:  When  would  you  like  it? 

YOU:  Well,  I  want  to  give  you  enough  time  to  do  a  good  job. 

RAM:  We’ll  do  a  good  job. 


A.4  04CDP1 :  JUST  TRYING  TO  HELP  (STORTI58) 

Word  Count:  138 

Intro:  You  work  for  a  U.S.  manufacturing  company  in  the  Middle  East.  A  while  back  you  asked 
Hassan,  a  local  employee,  if  he  knew  anyone  at  the  local  customs  office  who  might  be  able  to 
look  into  a  shipment  of parts  that  had  been  held  up  there  for  some  time. 


YOU: 

HASSAN: 

YOU: 


I  saw  the  man  in  the  Customs  Office  today. 

Oh  good. 

He  said  you  never  spoke  to  him  about  releasing  the  shipment. 


HASSAN:  I’m  very  sorry. 

YOU:  In  fact,  he  said  he’s  never  even  heard  of  you. 
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HASSAN:  It’s  possible. 

YOU:  But  when  I  asked  you  if  you  knew  him  and  could  help,  you  said  you  would  try. 

HASSAN:  Oh,  yes. 


YOU: 


But  it  wasn’t  true.  You  don’t  know  him  and  you  didn’t  talk  to  him. 


HASSAN:  Excuse  me.  But  I  was  only  trying  to  help. 

A.5  05CDP3:  OPTIONS  (STORTI19) 


Word  Count:  142 

Intro:  Ms.  Carrol,  an  American,  is  supporting  a  project  team  in  a  branch  of  a  U.S.  company  in 
the  Middle  East.  She  meets  regularly  with  Mrs.  Rafik,  the  local  branch  manager  to  plan  aspects 
of  an  important  project. 


MS.  CARROLL: 
MRS.  RALIK: 
MS.  CARROLL: 
MRS.  RALIK: 


There  are  several  ways  we  can  do  this. 

Yes.  I’m  sure. 

If  we  seek  donations,  we’ll  probably  get  them.  But  that  will  take  time. 
You’re  right. 


MS.  CARROLL:  If  we  spend  our  own  money,  we  can  start  right  away,  but  we’ll  have  to  go 

into  debt. 

MRS.  RALIK:  Most  probably. 


MS.  CARROLL:  Or  we  could  ask  for  an  advance  on  our  fourth-quarter  profits. 

MRS.  RALIK:  That’s  possible  too. 


MS.  CARROLL:  So  what  do  you  think? 

MRS.  RALIK:  I  think  we  should  pick  and  get  started. 


MS.  CARROLL:  But  we  have  to  decide  first. 

MRS.  RALIK:  Of  course. 


A. 6  06CDP3:  DEDICATION  (STORTI28) 

Word  Count:  154 

Intro:  Ms.  Lewis,  an  American,  is  the  managing  director  of  a  recently  acquired  branch  of  a  U.S. 
company  in  Italy.  One  day,  she  approaches  Mrs.  Fermi,  the  local  human  resource  manager,  to 
discuss  Mr.  Barzini,  the  branch ’s  aging  payroll  manager. 

MS.  LEWIS:  As  you  know,  we’ve  had  many  complaints  about  Mr.  Barzini. 

MRS.  LERMI:  What  kind  of  complaints? 

MS.  LEWIS:  He’s  very  slow  in  his  work  and  some  of  our  people  aren’t  getting  paid 

promptly. 
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MRS.  FERMI:  Yes.  Mr.  Barzini’s  been  with  us  a  great  many  years.  His  age  is  beginning  to 
catch  up  with  him. 


MS.  LEWIS:  So  you  agree? 

MRS.  FERMI:  Definitely.  After  so  many  years  of  dedicated  service,  we  can’t  expect  him  to 
perform  as  he  used  to.  We’ll  be  hiring  someone  new. 


MS.  LEWIS:  That’s  good  to  know.  How’s  he  taking  it? 

MRS.  FERMI:  Taking  it? 


MS.  LEWIS:  Losing  his  job,  I  mean. 

MRS.  FERMI:  Oh,  he’s  not  losing  his  job. 


A.7  07CDP3:  A  HELPING  HAND  (STORTI34) 

Word  Count:  124 

Intro:  Carl,  an  American,  is  a  supervisor  at  a  U.S.  manufacturing  plant  in  Mexico.  One  day  he 
observes  an  interaction  between  Juan,  a  local  crew  manager,  and  one  of  the  line  workers,  Raul. 

CARL:  Hey,  Juan.  Is  everything  OK? 

JUAN:  Yes,  sir.  I  was  just  explaining  to  Raul  here  about  the  new  drill  press.  Some  of  the  men 

aren’t  sure  about  it  yet. 

CARL:  I  know.  Actually,  I  overheard  you;  what  you  were  telling  Raul  isn’t  exactly  right. 

JUAN:  No? 

CARL:  No.  You  have  to  turn  the  fan  before  you  switch  on  the  water  jet,  not  after.  Now  try  it, 

Raul.  [Pause].  Yes.  That’s  it.  Any  more  problems  with  this,  Juan,  just  come  and  ask 
me.  That’s  what  I’m  here  for. 

JUAN:  Thank  you,  sir. 

A.8  08CDP3:  TWO  NEW  NURSING  STATIONS  (STORTI44) 

Word  Count:  121 

Intro:  Mr.  Compton,  an  American,  works  for  a  Non-Government  Organization.  He  travels  all 
over  the  Middle  East  advising  hospital  managers,  like  Mr.  Khouri,  on  how  to  administer 
donation  funds. 

MR.  KHOURI:  Would  you  like  to  see  the  two  new  nursing  stations? 

MR.  COMPTON:  Stations?  I  thought  we  agreed  to  build  one  station  and,  if  there  was  any 

money  left  over,  to  buy  some  equipment  for  it  so  we  could  start  using  it. 
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MR.  KHOURI: 

MR.  COMPTON: 

MR.  KHOURI: 

MR.  COMPTON: 
MR.  KHOURI: 


Yes,  but  there  was  enough  money  to  build  two  stations  at  once. 
But  is  there  any  money  left  over  to  equip  them? 

Unfortunately,  no. 

Then  we  can’t  use  them! 

Not  presently.  But  isn’t  it  good?  We  used  all  the  money. 


CIVILIAN  VIGNETTES 

A.9  09CV1P:  FOREIGN  BUREAUCRACY  (CUSH7) 

Word  count:  94 

You  have  recently  arrived  in  a  Middle  Eastern  country  and  obtained  a  position  as  a  private 
English  teacher. 

You  are  required  to  obtain  a  work  permit,  and  so  present  yourself  at  the  appropriate  government 
office  to  apply. 

You  are  told  to  fill  out  a  form  and  return  in  a  few  days. 

You  return  to  ask  if  the  permit  has  been  granted  and  are  told  the  officer  in  charge  has  not  had  a 
chance  to  sign  your  permit. 

You  are  asked  to  return  in  a  few  days. 

On  two  more  visits  you  meet  the  same  response. 


A.10  10CVP1 :  THE  WELCOMED  VISITOR  (CUSH21) 

Word  count:  159 

It  takes  some  time  getting  used  to — the  transition  to  living  on  a  small  Pacific  island  is  not  easy. 
You  are  the  only  American  the  locals  have  had  any  experience  with.  Except  for  the  occasional 
tour  boat  that  drops  eager  spenders  ashore  for  a  few  hours  at  a  time. 

At  first,  everything  seems  special.  Wherever  you  go,  people  rush  about  making  certain  you  are 
comfortable. 

The  best  welcome  mats  are  put  out  when  you  come  near,  the  best  food  is  made  available  to  you, 
and  everyone  seems  to  clean  their  homes  whenever  you  are  expected. 

The  villagers  seem  so  attuned  to  your  needs  that  you  are  usually  the  only  one  given  a  chair  to  sit 
upon. 

After  six  months  things  are  different.  You  are  not  showered  with  the  attention  to  which  you  have 
grown  accustomed.  People  seem  almost  indifferent  toward  you,  as  if  you  are  hardly  there. 
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A.11  11CVP1:  USING  THE  LOCAL  LANGUAGE  (CUSH33) 

Word  count:  171 
1st  Person:  Yes 

You  have  been  assigned  to  a  prospective  Asian  area  (in  a  country  undergoing  technological 
development)  to  do  a  field  site  study  of  the  region.  Excited  at  the  prospect  of  doing  well  on  your 
first  foreign  project,  you  bone  up  on  the  language  of  the  area. 

You  studied  the  language  in  college  and  did  very  well.  After  you  arrive  in  the  country,  you  begin 
immediately  to  talk  to  some  of  the  local  people  to  get  a  better  idea  of  the  area. 

Although  you  use  mostly  the  host  language,  you  notice  that  the  people  will  usually  giggle  and 
then  answer  you  in  English  even  if  they  only  know  a  little. 

Y ou  continue  talking  to  various  individuals  about  different  aspects  of  the  society. 

Often  when  you  are  trying  to  explain  a  relatively  complex  or  intricate  aspect  of  your  interest,  the 
people,  in  a  smiling  manner,  encourage  you  to  use  English. 

Even  when  you  are  confident  that  what  you  are  saying  is  correct,  people  laugh,  grin,  nod  their 
heads  and  then  encourage  you  to  continue. 


A.12  12CVP1:  BUSINESS  OR  PLEASURE  (CUSH61) 

Word  count:  236 

As  the  top  salesman  in  your  Midwestern  U.S.  area  you  are  asked  to  head  up  a  presentation  of 
your  office  equipment  firm  to  a  Latin  American  company.  You  have  set  up  an  appointment  for 
the  day  you  arrive,  and  even  begin  explaining  some  of  your  objectives  to  the  marketing 
representative  who  was  sent  to  meet  your  plane. 

However,  it  seems  that  the  representative  keeps  changing  the  subject;  he  persists  in  asking  a  lot 
of  personal  questions  about  you,  your  family,  and  your  interests. 

Y ou  are  later  informed  that  the  meeting  has  been  arranged  for  several  days  later,  and  your  hosts 
hope  that  you  will  be  able  to  relax  a  little  first  and  recover  from  your  journey,  perhaps  see  some 
sights  and  enjoy  the  country’s  hospitality. 

You  respond  by  saying  that  you  are  quite  fit  and  prepared  to  give  a  presentation  today,  if 
possible.  The  representative  seems  a  little  taken  aback  at  this,  but  says  he  will  discuss  it  with  his 
superiors. 

Eventually  they  agree  to  meet  with  you,  but  at  the  subsequent  meeting,  after  a  bit  of  chat  and 
some  preliminaries,  they  suggest  that  as  you  might  be  tired  they  can  continue  the  next  day  after 
you  have  had  some  time  to  recover. 

During  the  next  few  days,  you  notice  that  though  they  had  said  they  wanted  to  discuss  details  of 
your  presentation,  they  seem  to  spend  an  inordinate  amount  of  time  on  inconsequential  activities. 


A.13  13CVP3:  THE  UNSUCCESSFUL  DINNER  PARTY  (CUSH2) 

Word  count:  258 

Having  been  treated  to  a  wonderful  time  by  Mei-ying’s  family  on  her  first  visit  to  the  Orient, 
Alice,  an  American  exchange  student,  wants  to  return  their  hospitality.  She  invites  them  out  for  a 
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meal,  but  they  politely  refuse,  knowing  that  her  travel  budget  cannot  afford  it.  Being  aware  of  the 
Chinese  emphasis  on  food,  Alice  volunteers  to  make  the  family  a  genuine  American  meal.  They 
agree  to  this,  saying  that  they  will  get  whatever  she  needs.  Alice  makes  a  list,  and  Mei-ying  takes 
her  to  the  marketplace. 

There  seems  to  be  a  horde  of  people  pushing  and  grabbing  at  the  various  items  displayed  in 
every  available  spot,  right  there  in  the  street.  Mei-ying  attempts  to  maneuver  Alice  to  the  meat 
section,  where  she  can  get  some  steaks.  As  she  nears  the  area,  Alice  spots  a  man  who  is  scaling 
and  cleaning  out  a  large  fish  in  the  gutter.  At  this,  Alice  remarks  on  the  unsanitary  conditions  of 
the  place. 

She  nonetheless  makes  her  way  to  the  booth  with  the  meat.  Here  she  is  met  with  the  blank  stare 
of  a  dead  steer's  head.  Repulsed  at  this,  she  queasily  asks  Mei-ying  to  take  her  to  another  market, 
preferably  one  that  is  indoors. 

Mei-ying  hesitantly  agrees  saying  that  there  is  a  Western-style  supermarket  on  the  next  block  but 
that  she  rarely  goes  there. 

To  her  delight,  Alice  finds  all  the  items  she  needs.  However  she  notices  Mei-ying  poking  and 
pinching  and  squeezing  the  items. 

When  all  was  prepared  and  served,  Alice  noticed  that  Mei-ying’s  family  just  picked  at  the  food. 


A. 14  14CVP3:  BREAKFAST  AT  THE  CAFE  (CUSH57) 

Word  count:  100 

David  and  his  wife  Joanne  are  on  a  trip  to  southern  Europe. 

They  have  just  arrived  in  Spain  and  on  their  first  day  they  discover  a  little  cafe  near  their  hotel 
with  friendly  service  and  good  coffee. 

The  next  morning,  they  return  to  the  cafe  for  breakfast. 

They  sit  outside  in  the  warm  morning  sun  and  enjoy  an  excellent  breakfast  of  coffee  and 
croissants. 

However,  when  they  are  presented  with  the  bill,  David  notices  that  it  is  almost  double  what  they 
paid  the  day  before  for  exactly  the  same  items. 

Angrily,  he  calls  the  waiter  over  and  accuses  him  of  overcharging. 


A.15  15CVP3:  THE  SHINTO  PRIEST  (CUSH68) 

Word  count:  168 

The  U.S.  branch  of  a  Japanese  manufacturing  company  in  Japan  has  been  operating  successfully 
for  some  years,  but  in  recent  months  a  series  of  seemingly  unrelated  incidents  have  caused 
concern. 

First,  there  is  a  rash  of  accidents  in  the  plant  itself,  then  one  of  the  Japanese  executive’s  children 
dies  of  a  rare  illness  and  another  executive’s  car  catches  fire,  severely  burning  him. 

Rumors  of  a  jinx  on  the  company  begins  to  spread  among  the  employees,  and  morale  lowers. 
Consequently,  the  management  calls  a  meeting  of  the  executives  to  decide  how  to  react  to  the 
situation.  The  American  managers  suggest  that  all  safety  and  quality  control  procedures  be 
reviewed  so  as  to  reassure  the  workers  that  their  welfare  is  taken  seriously. 
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The  Japanese  managers,  however,  hold  that  this  has  already  been  done;  they  feel  that  other  forces 
are  at  work. 

They  wish  to  bring  in  a  Shinto  priest  to  bless  the  company  and  protect  it  against  evil  spirits — this 
is  the  only  course  of  action  that  will  reassure  them. 


A.16  16CVP3:  LEARNING  THE  ROPES  (CUSH71) 

Word  count:  145 
1st  Person:  Yes 

Helen  Conner,  an  American,  has  been  working  in  a  Japanese  company  involved  in  marketing 
cameras  for  2  years  and  is  well  respected  by  her  colleagues. 

In  fact,  she  is  so  respected  that  she  often  is  asked  to  work  with  new  employees  of  the  firm  as 
these  younger  employees  leam  the  ropes. 

Recently,  one  young  employee,  Hideo  Tanaka,  was  assigned  to  develop  a  marketing  scheme  for 
a  new  model  of  camera. 

He  has  worked  quite  hard  on  it,  but  the  scheme  is  not  accepted  by  his  superiors  because  of 
industry  wide  economic  conditions. 

Helen  and  Hideo  happen  to  be  working  at  desks  near  each  other  when  company  executives 
transmit  the  news  of  the  scheme’s  nonacceptance.  Hideo  says  very  little  at  that  point. 

That  evening,  however,  Helen  and  Hideo  happen  to  be  at  the  same  bar.  Hideo  has  been  drinking, 
and  he  vigorously  criticizes  his  superiors  at  work. 


MILITARY  VIGNETTES 

A.17  17MVP1 :  THIS  MAN  IS  A  JERK!  (MIL3) 

Word  count:  208 

You  are  a  Marine  Corps  Lieutenant  Colonel  deployed  to  Afghanistan  to  serve  as  an  advisor  to 
the  Commander  of  an  Afghan  National  Army  unit.  After  a  few  days  in  Kabul  you  are  transported 
to  the  Kandak  where  the  Afghan  unit  is  stationed.  Here  you  meet  the  commander  of  the  Afghan 
Battalion  for  the  first  time. 

After  brief  introductions,  the  Afghan  commander  shows  you  to  a  large  room  where  you  will  meet 
the  rest  of  the  Afghan  battalion  leadership  and  staff.  This  introductory  meeting  officially  marks 
the  replacement  of  the  Army  National  Guard  team  who  had  been  working  with  the  unit  prior. 

In  the  room  are  a  dozen  officers  of  the  battalion  scattered  about,  all  Afghans,  some  sitting  and 
some  standing. 

One  at  a  time  they  each  greet  you,  make  a  few  comments  and  praise  Allah.  Your  interpreter  is 
making  sure  you  are  apprised  of  everything  that  is  said,  whispering  his  comments  into  your  ear. 

It  gets  to  this  one  Afghan  officer  who  has  not  yet  been  introduced.  Your  interpreter  whispers  to 
you  that  this  is  the  unit’s  religious  officer. 

The  religious  officer  stands  up  and  like  the  others  first  says,  “Praise  be  to  Allah...”  and  then  he 
points  his  finger  at  you  and  exclaims,  “You  are  a  jerk.” 
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A.18  18MVP1 :  THE  COLA  BOTTLE  (MIL10) 

Word  count:  248 

You  are  a  Marine  Corps  Major  serving  as  company  commander  in  Somalia.  On  one  of  your  first 
ventures  outside  of  the  base  you  meet  the  leaders  of  a  local  village  who  have  never  engaged  with 
the  U.S.  military  before.  You  can  tell  that  the  meeting  is  a  big  deal  for  the  local  leadership  as 
they  all  appear  to  have  donned  their  best  clothes. 

One  of  the  village  elders  is  particularly  eager  to  show  you  around  in  the  little  town,  his  little 
village.  You  quickly  agree  and  before  you  set  out  the  elder  grabs  a  Cola  out  of  a  cooler  and 
hands  it  to  you.  You  notice  that  the  Cola  is  cold;  the  little  cooler  has  ice  in  it. 

The  two  of  you  walk  through  the  village  and  after  a  little  while  you  arrive  at  a  makeshift  medical 
clinic  that  has  recently  been  stood  up  by  a  non-government  group.  Beaming  with  pride  the  elder 
shows  you  around  the  group  of  tents  that  make  up  the  clinic. 

All  of  a  sudden  the  elder  sees  that  you  are  standing  there  with  your  Cola  unopened.  He  quickly 
grabs  the  Cola  and  reaches  into  a  medical  bin  full  of  medical  equipment. 

You  watch  this  unfold  quickly,  but  immediately  notice  the  bloody  gauze  in  the  bin  and  recognize 
that  the  equipment  in  it  has  been  used  for  medical  procedures. 

The  elder  selects  an  instrument  from  the  bin,  shakes  it  off  and  uses  it  to  open  the  bottle.  He  then 
hands  the  Cola  to  you. 


A. 19  19MVP1 :  THE  MULLAH  (MIL25) 

Word  count:  215 

You  are  an  Army  Captain  leading  a  US-Afghan  team  that  is  conducting  a  poppy  clearing 
operation  in  the  Tagab  valley.  As  part  of  the  operation  you  provide  humanitarian  assistance  to 
villages  along  the  way. 

A  Mullah  has  been  working  with  the  Americans  on  the  team,  and  he  has  been  very  helpful.  He 
goes  in  to  the  villages  and  talks  about  the  poppy  clearing,  and  all  the  things  the  team  will  do  to 
help  the  villagers. 

For  the  humanitarian  assistance,  the  team  will  talk  to  locals,  talk  to  elders,  and  distribute  rice,  oil, 
school  kits  for  kids,  soccer  balls,  prayer  mats,  and  other  items.  They  have  scheduled  a  number  of 
these  events  during  the  road  building,  and  it  has  been  going  very  well. 

At  the  end  of  the  road  building  operation,  the  U.S.  team  and  Afghan  partnered  forces  are  packing 
everything  up.  Your  Afghan  interpreter  comes  up  and  says  "Hey  sir,  there's  a  lot  of  stuff  left 
over.”  “Really?”  you  say.  “They  said  they  distributed  it  all." 

The  interpreter  says  "Yeah,  they  kept  some;  they're  hiding  it  in  that  truck." 

The  Afghan  leader  there  at  that  time  is  the  Mullah.  You  go  to  him  and  say  "I  understand  we  have 
supplies  left  over."  The  Mullah  replies  "No,  no  we  don't  have  supplies.  They're  all  distributed." 


A.20  20MVP1 :  RIDING  THE  BUS  (MILW1) 

Word  count:  286 
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You  have  just  been  assigned  as  the  10  officer  for  an  area  in  Kosovo.  The  overall  mission  is  to 
maintain  security.  In  this  area,  the  Serbs,  a  protected  minority  in  Kosovo,  live  in  enclaves  and  are 
afraid  to  leave  these. 

In  an  outbrief,  the  previous  officer  provided  information  on  a  Serb  bus  situation  in  your  area. 
Two  years  before  your  arrival,  a  bomb  blew  up  a  bus,  and  killed  17  students.  Therefore,  the  U.S. 
has  been  escorting  Serb  college  students  to  school  placing  an  armored  vehicle  in  the  lead  and  in 
trail  of  their  bus.  About  60-80  students  use  this  bus  every  day. 

But,  it  is  very  expensive  to  provide  the  full  escorts  and  a  decision  is  to  increase  their  efficiency 
while  maintaining  security.  First,  the  two  armored  vehicles  will  be  reduced  to  1  for  a  period,  and 
then  down  to  zero  but  with  a  guard  on  the  bus. 

The  thinking  is  that  a  demonstration  of  additional  UAV  air  surveillance  and  security,  along  with 
the  motivation  of  the  students  to  finish  the  term,  will  carry  them  through.  Some  drop  in  ridership 
is  expected,  though  it  is  assumed  over  time  the  levels  will  go  back  up. 

The  planned  changes  are  described  to  the  students  and  they  are  shown  videos  from  the  UAVs 
presenting  aerial  views.  The  students  are  told  to  wave  at  the  UAVs  so  they  can  see  themselves  on 
the  monitors  to  demonstrate  the  quality  of  the  surveillance. 

About  two  weeks  after  the  announcement,  the  2  escorts  are  reduced  to  one.  The  ridership  drops 
to  50-60  students/day.  After  two  weeks,  the  remaining  escort  is  removed  and  a  lightly  armed 
guard  is  placed  on  the  bus.  The  number  of  riders  drops  down  to  10  students  a  day. 

A.21  21MVP3:  ANOTHER  SEAT  ON  THE  HELICOPTER  (MIL11) 

Word  count:  269 

U.S.  Marine  Corps  Major  Franks  is  deployed  to  Afghanistan  with  a  civil  affairs  group  attached  to 
a  provincial  reconstruction  team.  Franks  often  accompanies  the  Provincial  Governor  and  his  staff 
on  visits  to  Shurras  and  meetings  in  towns  and  villages  all  over  the  district. 

To  get  to  these  meetings  the  travel  party  always  makes  use  of  a  U.S.  Army  helicopter  that  has 
been  made  available  to  the  local  Afghan  leadership.  This  helicopter,  however,  has  a  limited 
number  of  seats;  a  little  more  than  20. 

The  first  time  the  Provincial  Governor  and  his  party  arrive  at  the  airfield  it  becomes  clear  that  not 
everyone  will  fit.  Franks  informs  the  Governor’s  head  security  officer  of  the  problem,  “You  have 
20  seats  to  give  out  to  people.  You  choose.”  When  it  comes  time  to  get  on  the  helicopter,  Franks 
stands  by  the  side  of  the  helicopter  along  with  the  Afghan  security  officer  and  counts. 

They  get  to  a  point  where  the  police  chief,  who  is  a  Colonel,  and  his  two  body  guards  are  still 
waiting  to  board  and  there  is  only  room  for  one  more.  The  governor  and  his  body  guards  are 
already  seated  in  the  helicopter.  “It  looks  like  there  will  only  be  room  for  you  today,  sir?”  Franks 
says. 

“I  want  my  two  body  guards  to  go  also,”  the  police  chief  retorts.  Franks  is  firm,  “no,  you  can’t 
bring  them.  We  don’t  have  room  for  them.”  The  two  argue  back  and  forth  for  a  while  before  the 
police  chief,  although  visibly  upset,  complies. 

Six  weeks  later  the  exact  same  thing  happens.  In  fact,  this  repeats  every  time  the  Provincial 
Governor  travels. 
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A. 22  22MVP3:  BITTER  ALMONDS  (MIL17) 

Word  count:  276 

U.S.  Army  Staff  Sergeant  Waterman  is  deployed  as  a  human  intelligence  collector  in 
Afghanistan.  One  of  the  local  leaders  that  Waterman  is  assigned  to  meet  with  on  a  regular  basis 
is  named  Wahidi.  Waterman  is  eager  to  develop  a  relationship  with  Wahidi  so  he  takes  care  to 
make  sure  that  that  tea,  dried  fruit,  and  nuts  are  available  whenever  they  meet  to  help  break  the 
ice.  Waterman  himself  is  particularly  fond  of  the  Afghan  spiced  almonds. 

Even  so,  developing  a  relationship  is  slow  going.  Over  the  course  of  their  meetings  Waterman 
tells  Wahidi  about  his  background,  his  family  and  hometown.  In  their  third  meeting  Waterman 
says,  “Let  me  leam  something  about  you.” 

Wahidi  replies,  “Well,  what  do  you  want  to  learn  about  me?  I’m  Afghan  and  I’ve  been  in  this 
country  all  my  life,  I  don’t  know  anything  else.” 

Waterman  is  not  disheartened,  “Well,  tell  me  about  your  history  and  the  war  with  the  Russians.” 
Wahidi  shakes  his  head  and  says,  “You  know.  I  have  a  question  to  ask  you  but  I  don’t  want  you 
to  be  offended.”  Waterman  is  intrigued,  “You  can  ask  me  anything”  he  says.  Wahidi  continues, 
“Why  do  you  always  have  the  almonds  in  front  of  you?” 

Waterman  looks  at  the  bowl  in  front  of  him.  “I  don’t  know,”  he  says.  Wahidi  cocks  his  head, 
“Do  you  know  what  almonds  are  considered  in  Afghanistan?”  Waterman  is  mystified,  “No.” 
“We  call  them  brain  food,  food  for  the  brain.” 

“Okay,”  Waterman  replies  slowly.  “Every  time  I  meet  you’re  the  only  one  eating  the  almonds. 
And  so  I’m  thinking  you’re  trying  to  be  smarter  than  I  am  because  you  never  offer  me  any,” 
Wahidi  explains. 

A.23  23MVP3:  IS  THAT  A  “NO”?  (MIL19) 

Word  count:  238 

Tensions  within  a  large  country  in  the  Middle  East  have  been  on  the  rise.  The  international 
community  and  U.S.  leadership  are  worried  that  the  brewing  conflicts  between  warring  factions 
within  the  country  might  soon  lead  to  an  all-out  civil  war.  U.S.  Marine  Corps  General  VanLehn 
accompanies  prominent  U.S.  State  representative,  Mr.  McDonald,  on  official  visits  to  leaders  in 
in  the  Middle  Eastern  states  surrounding  the  country  in  question.  The  purpose  of  the  visits  is  to 
ask  permission  to  use  their  military  bases  as  staging  grounds  in  preparation  for  the  possibility 
that  the  situation  escalates. 

In  Saudi  Arabia  VanLehn  and  McDonald  meet  with  Chief  of  the  Armed  Forces  of  Saudi  Arabia. 
The  meeting  goes  well.  The  three  of  them  along  with  a  few  of  the  official’s  staff  and  an 
appointed  interpreter  have  tea.  They  discuss  history  as  well  as  current  political  events. 

The  Saudi  military  chief  seems  to  share  their  concerns  about  the  stability  of  the  situation  in  the 
neighboring  country. 

When  towards  the  end  of  the  meeting  McDonald  asks  the  chief  about  opening  up  his  military 
bases  to  the  U.S.  and  international  community  the  chief  changes  the  subject. 
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After  they  formally  conclude  the  meeting  the  chief  follows  them  to  the  door.  As  they  walk  out  he 
puts  his  hand  on  VanLehn’s  shoulder  and  says,  “Remember,  in  the  end  we’re  always  your 
friends.” 

When  they  drive  back  to  their  hotel,  McDonald  says,  “I  guess  we  can’t  use  their  bases.” 


A.24  24MVP3:  THE  VILLAGERS  (MIL23) 

Word  count:  261 

U.S.  Army  Captain  Lang  is  leading  a  humanitarian  assistance  operation  in  a  village  south  of 
Kabul.  It  is  a  joint  operation  between  Lang’s  team  and  an  Afghan  National  Army  unit.  The  team 
drives  to  the  village  to  deliver  clothing  supplies  and  blankets. 

Someone  has  informed  the  local  Police  Chief  of  the  operation,  and  the  Police  Chief  provides  an 
escort  of  four  or  five  cars.  As  the  team  arrives,  it  seems  as  though  the  entire  village  comes 
running  up  to  them. 

The  team  is  not  even  set  up  yet,  and  people  are  crowding  around,  pushing  each  other,  grabbing 
for  the  clothing  and  blankets.  Lang  notices  that  the  clothes  people  are  wearing  are  clean  and  in 
good  condition,  so  he  thinks  they  are  probably  not  the  most  poor. 

The  team  starts  noticing  that  they  have  given  items  to  the  same  people  from  the  village  two  or 
three  times.  The  same  villagers  come  up  wearing  different  hats  or  clothes,  and  each  time  they 
take  as  much  as  they  can  possibly  carry. 

This  leads  to  a  shortage  of  supplies,  and  the  villagers  that  are  slower  getting  there  do  not  get 
anything.  The  team  then  has  to  stop  people  from  fighting  each  other  for  the  items.  Even  people 
who  have  arrived  together  in  the  same  vehicle  fight  each  other  for  supplies. 

The  team  also  realizes  that  even  some  of  the  local  police  who  are  part  of  the  escort  are  lining  up 
to  receive  items.  They  have  changed  out  of  their  uniforms,  put  on  civilian  clothes,  and  joined  the 
crowd  of  villagers  around  the  distribution  truck. 
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APPENDIX  C:  EMOTION  REGULATION  VIGNETTES,  LONG- 
FORM 

1)  Dowry  money  &  Child-brides  (the  dowry  money  and  child-bride  can  be  separated; 
technically  in  many  rural  parts  there  is  still  dowry  money  even  for  somewhat  more 
"normal"  marriages). 

The  elders  of  two  families  come  to  an  agreement  regarding  a  particular  marriage.  A  teenage  girl 
(even  as  early  as  12)  is  married  off  to  an  older  man  (could  either  be  around  late  20s,  late  30s  or 
much  older  in  their  50s,  60s-in  the  latter  case,  as  a  second  wife).  In  return  the  guy’s  family  pays 
the  girl’s  family.  The  dowry  can  be  in  terms  of  cash  ($  even  better!),  sheep  etc.) 

2)  Different  types  of  Marriage 

Another  kind  of  marriage  is  that  in  the  case  of  death  of  a  brother,  where  the  widowed  sister-in 
law  is  wed  with  a  single  or  widower  brother-in  law  (Levirat).  This  is  done  in  order  to  keep  the 
inheritance  of  the  deceased  in  the  family  and  for  the  children’s  welfare.  Similarly,  it  is  also 
possible  that  a  sister  gets  married  to  a  deceased  sister’s  husband  (Sororat). 

In  the  past,  particularly  in  rural  areas,  men  would  “snatch”  or  kidnap  a  girl  that  they  wanted  to 
marry,  but  were  not  allowed  to  for  one  reason  or  the  other,  mostly  due  to  objection  by  families.  It 
also  happened  that  a  girl  and  boy  run  away  in  mutual  agreement  to  overcome  family  objection  to 
their  matrimony.  In  some  regions,  there  is  also  a  type  of  “snatching”  called  “oturakalma”,  where 
a  girl  goes  to  the  home  of  the  man  that  she  loves  and  settles  there,  many  times  at  the  objection  of 
her  own  family.  -  THIS  IS  RATHER  COMMON 


Another  type  of  marriage  which  was  often  practiced  in  the  past  is  “be§ik  kertme”.  While  children 
were  yet  in  their  cradle,  they  were  promised  or  “engaged”  to  each  other  by  their  families.  In  the 
past,  the  rejection  of  marriage  by  the  girl  or  boy  once  they  reach  marrying  age,  would  be 
regarded  as  dishonor  to  the  family  and  even  could  cause  bloody  family  feuds.  This  type  of 
arranged  marriage  is  almost  extinct  today. 


3)  Having  kids 

Two  sisters.  Both  married.  One  of  them  cannot  have  kids  for  whatever  reason.  When  the  sibling 
has  a  child,  her  kid  may  be  registered  under  the  other  sister.  Either  /Both  parties  may  be  involved 
in  the  bringing  up  of  the  kid.  (Probably  less  common  nowadays,  but  more  so  2  generations  up). 

4)  Prayer  time: 

Men  go  to  the  mosque  right  before  prayer  time,  and  at  the  mosque,  there  are  fountains  (outside!). 
Men  sit  there  and  take  off  their  shoes,  socks  etc.  and  wash  up  in  public  prior  to  namaz  (ablution) 

5)  Prayer  time- 

Friday  noon  prayers  are  special  in  Islam-some  believers  would  argue  that  if  you  miss  a  few  of 
these  you  would  end  up  in  Hell.  Also,  it  is  required  that  you  pray  as  a  group  in  a  mosque 
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(cemaat).  This  results  in  people  putting  up  their  prayer  mats  in  streets  when  they  cannot  find 
room  in  the  courtyard  in  the  mosque.  This  can  1)  block  traffic  -  but  you  cannot  say  anything—  2) 
social  pressure  to  the  shop-owners  to  close  their  shops  because  people  (especially  women)  are 
not  supposed  to  walk  by  the  men  when  the  men  are  praying. 

6)  Being  affectionate  in  public  &  beating. 

A  couple  sits  hands  in  hand  on  the  bus,  and  then  the  bus  driver  stops  the  bus,  basically  beats 
them  off  the  bus,  saying  "This  is  not  a  place  for  sex".  Another  passenger  saying  that  the  couple 
was  not  disruptive  was  then  also  forced  to  get  off  the  bus. 

7)  "Koceks" 

Koceks  are  known  to  be  male  dancers  entertaining  males.  They  would  dress  up  like  woman,  but 
not  like  transvestites.  Does  not  have  the  sexual  connotation.  Purely  entertainment  purposes. 

In  the  Turkish  culture  there  are  male  dancers,  called  koceks-  this  tradition  has  its  roots  in 
Ottoman  era;  nowadays  it  is  considered  more  folkloric.  A  guy  dresses  up  as  a  dancer,  dances  and 
sings/entertains  etc.,  and  other  men  watch  him  perform. 

The  Radio  and  Television  Supreme  Council  in  Turkey  banned  the  television  reality  program 
"He's  a  Lady  Now,"  in  which  men  would  compete  to  best  dress  and  act  like  a  woman.  According 
to  the  report,  the  program  was  banned  following  an  alleged  public  protest. 

8)  Men-"buddies"  walking  arm-in-arm  and  hugging/kissing  in  public  /  dancing;  no  sexual 
implication. 

9)  Circumcision  ceremony 

There  are  many  things  about  the  circumcision  business  that  may  be  worked  into  a  vignette: 

1.  outfit  —  and  how  kids  are  made  to  walk  in  public  in  these  outfits— more  funny  than 
disgusting 

2.  no  anesthesia  &  the  operation  (a  family  elder  holds  the  kid  tight  and  people  chant  as  he  is 
circumcised) 

3.  public  circumcision  ceremonies  where  municipalities  have  hundreds  of  kids  circumcised 
at  stadiums 

4.  foreign  groom  gets  circumcised  because  of  family  pressure 

The  family  starts  to  prepare  the  child  for  the  circumcision  a  few  days  before  the  ceremony.  In 
fact,  the  child  begins  to  experience  the  joy  and  the  fear  of  the  operation  long  before  that.  In 
traditional  communities,  parents  start  to  prepare  their  son  for  this  significant  turning  point  in  life 
months  beforehand. 


A  special  circumcision  outfit  is  the  most  important  part  of  the  preparations  for  the  ceremony. 
Rich  families  adorn  their  sons  with  jewels  in  big  cities,  and  a  light  blue  headgear  on  which  the 
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word  “Ma§allah”  is  embroidered  on  the  front  is  the  most  common  element  of  circumcision 
outfits  there.  In  villages,  the  children  who  will  be  circumcised  wear  new  outfits;  a  “9evre”  (a 
surrounding  piece  of  cloth)  and  “yaglik”  (a  large  napkin)  are  hung  around  their  necks  and 
shoulders,  and  bridal  tinsel  is  hung  from  the  back  of  their  headgear.  A  few  days  before  the 
ceremony,  or  even  on  the  day  itself,  the  children  to  be  circumcised  are  paraded  around  with  their 
friends,  either  on  horseback,  on  a  cart  or  else  by  car.  People  are  informed  by  means  of  this 
parade  that  the  child  is  to  be  circumcised. 


The  circumcision  procedure  consists  of  cutting  off  the  child’s  foreskin.  The  boy  is  sat  on  the  lap 
of  his  kirve  (someone  who  acts  as  a  kind  of  godfather  at  the  circumcision)  if  he  has  one,  or  if  not, 
on  the  lap  of  some  other  relative.  The  boy  is  made  to  open  his  legs,  and  the  person  whose  lap  he 
is  sitting  on  holds  the  boy’s  arms  very  tight.  Meanwhile,  encouraging  words  stressing  manly 
virtues  are  spoken  in  order  to  help  reduce  the  child’s  fears.  Before  and  during  the  operation,  the 
words  “Allahu  ekber  Allahu  ekber”  are  uttered,  and  people  recite  a  very  common  bantering 
couplet  “oldu  da  bitti  ma§allah,  iyi  olur  in§allah”  (It  has  happened  at  once,  May  God  preserve 
him;  it  will  grow  better,  by  God’s  will).  The  person  who  carries  out  the  circumcision  procedure; 
that  is,  the  operation,  is  generally  referred  to  a  “siinnet9i”,  meaning  circumciser.  This  individual 
is  also  known  as  the  “abdal”  (wiseman)  or  “kizilba§  abdal”  (scarlet-head  wiseman)  in  Central 
and  Eastern  Anatolia. 


10)  Woman  Beating  /  Killing  /  Honor-killings 

A  young  girl  is  raped,  and  to  clean  the  family's  honor/name  she  is  murdered  by  her 
brothers/uncles/father....  (many  variants  of  this  is  possible).  Typically  the  family  won’t  take  the 
body,  so  these  girls  are  buried  by  members  of  NGOs  to  homeless  cemeteries. 

11)  Abortion 

More  recent  stuff-the  government  pushed  a  ban  on  abortions-  it  is  still  legal,  but  the  doctors  can 
refuse  it,  a  lot  of  pressure  against  it.  Worse  case  it  some  hospitals  have  started  to  perform 
abortions  without  anesthesia  and  belittling  woman  throughout  the  surgery,  treating  them  as 
whores  etc. 

12)  Police  violence 

Police  officer  attacked  a  protestor  (  a  highschool  student)  and  aimed  and  shot  a  teargas  bullet  in 
close  range  injuring  a  girl,  17  years,  hospitalized  &  coma.  They  did  this  after  they  pulled  her  out 
from  the  building  she  was  hiding. 

13)  Imam  &  sex  &  religious  marriage 

•  The  Islamic  version  of  priest  &  child  molestation  sex  scandal. 

•  People  get  a  "religious  marriage"  by  an  imam  for  a  few  hours/days  to  basically  have  sex. 

•  *A  college  prof,  gets  a  religious  marriage  with  a  student  to  force  her  into  sex,  blackmails 
her  etc. 
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APPENDIX  D:  APPROVAL  FROM  ONR  TO  CONDUCT  STUDY 
3 


From:  Kim,  Joong  H  CIV  ONRA,  30  <joong.kim@navy.mil> 

Sent:  Tuesday,  April  28,  2015  3:39:44  PM 
To:  Chevillet,  Mark  A. 

Cc:  Kollmorgen  Gary  (gkollmorgen@gskinc.net);  Merkle,  Andrew  C.;  Mastroianni,  Lee  CIV  ONR,  30 
Subject:  RE:  N00014-12-1-0629_REQUEST  FOR  A  NO  COST  EXTENSION 

The  approval  has  been  submitted  with  the  POP  end  date  changed  to  Dec  31  2015.  Please  proceed  with 
the  new  plan.  I  still  expect  the  project  to  be  concluded  by  the  end  of  the  fiscal  year. 

v/r, 

Joong  Kim,  PhD 
Program  Officer,  Code  30 
Office  of  Naval  Research 
Ph:  703-696-4742 
Email:  joong.kim@navy.mil 
SIPR:  joong.kim@navy.smil.mil 

. Original  Message . 

From:  Chevillet,  Mark  A.  [mailto:Mark.Chevillet@jhuapl.edu] 

Sent:  Tuesday,  April  28,  2015  3:26  PM 
To:  Kim,  Joong  H  CIV  ONRA,  30 

Cc:  Kollmorgen  Gary  (gkollmorgen@gskinc.net);  Merkle,  Andrew  C. ;  Mastroianni,  Lee  CIV  ONR,  30 
Subject:  Re:  N00014-12-1-0629_REQUEST  FOR  A  NO  COST  EXTENSION 

Hello  Dr.  Kim, 

We  are  sympathetic  to  your  request  (sent  recently  via  Mr.  Kollmorgen)  to 
proceed  with  spending  on  schedule,  but  are  unsure  whether  to  proceed  with 
our  altered  plans  prior  to  receiving  your  approval  (the  original  planned 
experiments  cannot  be  conducted  in  this  case  due  to  their  need  for  human 
subjects  research  approval).  Per  your  request,  we  provided  the  outline 
included  below  describing  the  proposed  changes  to  the  experimental  plan 
(to  accommodate  the  decision  not  to  pursue  human  subjects  research 
approvals)  and  I  wanted  to  check  in  to  see  whether  you'd  had  a  chance  to 
review  it?  We  would  be  happy  to  discuss  if  it  would  be  helpful. 

Please  let  me  know  if  you  would  like  us  to  proceed  with  the  revised  plans, 
or  await  your  approval  before  proceeding,  and/or  whether  you'd  like  us  to 
arrange  for  a  time  to  discuss  either  in  person  or  on  the  phone. 

Best, 

Mark 


Mark  A.  Chevillet,  Ph.D. 

Program  Manager,  Applied  Neuroscience 
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Research  and  Exploratory  Development  Department  ohns  Hopkins  University 
Applied  Physics  LaboratoryPhone:  443-778-3514  (Baltimore)  /  240-228-3514 
(Washington) 


On  4/17/15,  5:11  PM,  "Chevillet,  Mark  A."  <Mark.Chevillet@jhuapl.edu> 
wrote: 

>Hi  Dr.  Kim, 

> 

>My  sincere  apologies.  I  was  unsure  of  the  correct  procedure,  but  should 
>have  ensured  we  touched  base  with  you  before  sending  this  request. 

> 

>Please  let  me  know  if  you  need  something  more  formal,  or  if  you  would  like 
>to  discuss  directly,  but  an  outline  of  the  re-planning  from  the  PI 
>(Jonathan  Kopecky)  is  included  below.  Should  you  choose  to  grant  the 
>extension,  the  end  of  the  fiscal  year  would  allow  for  a  smooth  execution 
>of  the  proposed  plans. 

> 

>Please  let  me  know  if  I  can  provide  any  additional  information. 

> 

>Best, 

> 

>Mark 

> 

>Mark  A.  Chevillet,  Ph.D. 

>Program  Manager,  Applied  Neuroscience 
>Research  and  Exploratory  Development  Department 
>Johns  Hopkins  University  Applied  Physics  Laboratory 
>Phone:  443-778-3514  (Baltimore)  /  240-228-3514  (Washington) 

> 

> 

> 

>The  neural  correlation  of  cross-cultural  adaptation  is  an  ONR-sponsored 
>project  designed  to  investigate  the  culture-general  skills  and  traits 
>needed  to  succeed  in  interacting  successfully  with  people  from  other 
>cultures.  The  goal  of  this  project  was  to  better  identify  these  skills  (so 
>we  could  better  train  these  skills)  or  these  traits  (so  we  could  better 
>select  personnel  exhibiting  those  traits).  As  part  of  this  project,  there 
>were  two  proposed  experiments.  The  first  was  an  online  behavioral 
experiment  assessing  personality  and  psychometrics  measures,  to  be 
>correlated  with  subsequent  peer  recommendations.  The  second  was  an  fMRI 
>study  that  examined  the  neural  correlates  of  high  and  low- performers  in 
>our  behavioral  assessment  to  see  if  there  were  differences  in  emotion 
regulation  and  non-verbal  communication  ability. 

> 

>ln  November,  2014,  we  received  notice  that  the  HRPO  approval  request  would 
>not  be  submitted  due  to  time  and  cost  associated  with  the  process  having 
>not  been  previously  planned.  As  the  two  planned  experiments  both  required 
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>HRPO  approval,  neither  could  then  be  executed.  While  addressing  this 
>issue,  we  came  into  contact  with  The  Army  Special  Forces  and  their  Special 
>Forces  Assessment  and  Selection  course  (SFAS),  who  are  also  interested  in 
>cross-culture  adaptation.  Because  research  in  this  domain  is  limited,  they 
>were  unsure  what  skills  to  train  or  what  traits  best  predict  success.  As 
>part  of  their  process,  they  collect  and  maintain  data  on  the  behavioral 
>and  physical  assessments  of  SFAS  participants  and  had  access  to  roughly 
>6000  participants.  The  SFAS  group  offered  to  pull  together  these  data  and 
>de-identify  them,  in  which  case  their  analysis  would  not  qualify  as  human 
>subjects  research.  They  are  currently  in  the  process  of  collecting  and 
>de-identifying  these  data  to  provide  to  J  HU/APL.  We  propose  to  analyze 
>these  data  in  the  hopes  of  quantitatively  determining  which  measures  best 
>predict  success  in  SFAS. 

> 

>The  data  includes  many  measures  related  to  culture-general  skills, 
including  the  defense  language  acquisition  battery  scores  (DLAB),  defense 
language  proficiency  test  scores  (DLPT),  various  measures  of  IQ,  various 
>measure  of  physical  fitness,  and  certain  measures  that  combine  the  two, 
>such  as  assessments  on  a  test  that  combine  navigation  and  speed.  For  each 
participant,  personality  metrics  and  peer  assessments  were  also  included. 
>Should  it  be  possible  to  sufficiently  de-identify  those  data,  they  would 
>be  provided  for  correlational  analyses  as  well.  All  data  were  previously 
>collected  (during  2013  and  2014)  and  are  to  be  de-identified  before  ever 
>being  provided  to  J  HU/APL. 

> 

>Data  are  expected  to  be  available  within  the  next  month,  leaving  little 
>time  to  analyze  them  before  the  current  end  of  the  Period  of  Performance 
>in  J  une.  Should  it  be  possible  to  extend  the  period  of  performance  to  the 
>end  of  the  fiscal  year,  these  data  could  be  thoroughly  analyzed  and  the 
>results  properly  documented  as  a  concrete  outcome  of  the  neural 
correlation  of  cross-cultural  adaptation  project.  Your  consideration  of 
>this  request  is  greatly  appreciated. 


APPENDIX  E:  IRB  PROPOSAL 

1.  Abstract 

What  contributes  to  an  individual’s  success  in  work  situations  demanding  adapting  to 
various  cross-cultural  circumstances?  In  what  way  do  emotions  and  stress  impact 
reasoning,  decision  making,  interpretation  of  behaviors  and  judgment  of  other’s  intention 
in  such  work  situations?  It  is  here  asserted  that  there  are  three  main  contributors  to 
expertise  in  cross  cultural  adaptability:  1)  cultural  sensemaking,  2)  moral  reasoning 
capability,  and  3)  social  mindreading  skill.  In  Part  1  of  this  two-part  study,  we  will 
conduct  an  examination  of  cognitive,  emotional,  and  social  factors  that  are  likely  to 
underlie  expertise  in  these  three  contributors,  and  through  them  to  cross-cultural 
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adaptability  in  general.  Specifically,  we  will  administer  a  series  of  self-report  assessments 
to  participants  experienced  with  and  competent  in  adapting  across  cultures  (experts),  and 
a  novice  comparison  group  of  participants  who  lack  experience  in  cross-cultural 
adaptation.  These  data  will  be  collected  via  an  on-line  survey  web  site.  As  a  result  of  Part 
1,  we  will  be  able  to  characterize  the  trait-  and  performance-based  attributes  and  skills 
associated  with  cross-cultural  expertise.  Having  then  determined  the  behavioral 
characteristics  of  cross-cultural  expertise,  we  will  proceed  to  the  Part  2  study  whose 
purpose  is  to  examine  the  underlying  neural  mechanisms  of  cross-cultural  expertise  as 
related  to  moral  reasoning  and  social  mindreading. 

2.  Objectives 

The  primary  objectives  of  the  present  study  are: 

•  Contrast  cross-cultural  experts  (individuals  who  are  experienced  with  and  skilled  in 
interacting  across  cultures)  with  novices  (those  individuals  who  are  less  experienced 
with  cross-cultural  challenge). 

•  Determine  the  set  of  trait-  and  perfonnance-based  behavioral  assessments  that 
discriminate  the  expert  and  novice  cohorts. 

•  Confirm  and  explore  the  competence,  approach,  and  interpretation  in  moral  reasoning 
challenge  as  observed  in  experts  and  novices. 

•  Establish  and  determine  a  differential  characterization  of  the  expert  and  novice  in 
tenns  social  mindreading  skill. 

3.  Background 

There  is  considerable  civilian  and  military  interest  in  determining  which  individuals  are 
best  suited  for  working  abroad.  Much  of  this  research  focuses  on  long-term 
acculturation,  which  is  most  applicable  when  migrating  to  one  particular  culture  (see,  for 
example,  Berry,  et  al.,  2002).  However,  for  certain  types  of  jobs,  individuals  are  required 
to  go  to  multiple  locations  and  spend  a  relatively  short  amount  of  time  (on  the  order  of 
months)  at  each  place  thereby  prohibiting  integration  and  familiarization  with  that  native 
culture.  Despite  the  constraints  of  the  length  of  time  within  each  cultural  environment, 
the  assignments  demand  extensive  and  often  high  stake  interactions  with  locals.  These 
circumstances  can  be  difficult  and  are  often  emotionally  charged  and  fragile  (Goodwin, 
2005).  There  are  different  issues  involved  in  this  type  of  short-term  scenario,  to  which  we 
refer  as  “cross-cultural  adaptation,”  versus  long-term  migration. 

For  some  short- duration  missions,  general-culture  competency  and  skills  such  as 
knowledge  about  the  typical  schedule  and  food  preferences  are  sufficient.  However,  there 
are  short-term  missions  for  which  these  general-culture  skills  are  insufficient  and  culture- 
specific  skills  are  required  to  assure  that  one’s  mission  is  accomplished.  For  example,  the 
skills  required  for  a  Francophile  to  successfully  immigrate  and  integrate  into  French 
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culture  is  very  different  than  the  skillset  required  to  work  in  France  for  three  weeks,  work 
in  Italy  for  two  months,  then  consult  in  China,  etc.  Living  and  working  in  so  many 
different  cultures  can  be  particularly  stressful,  as  one  never  has  a  chance  to  acculturate 
sufficiently  to  feel  comfortable  in  that  particular  culture.  It  also  requires  the  ability  to 
assimilate  information  quickly,  not  only  to  leam  about  the  culture  rapidly  but  also  to 
adjust  one’s  knowledge  of  the  culture  as  one  interacts  with  people  from  that  culture. 

A  general  framework  for  cross-cultural  psychology  (Vaughn  &  Phillips,  2009)  has 
identified  three  major  requirements  for  successful  cross-cultural  interactions: 
intrapersonal  competence,  interpersonal  competence,  and  cultural  competence. 
Intrapersonal  competence  refers  to  the  knowledge  of  one’s  self  and  individual  mental 
skills  such  as  self-control,  openness  to  new  situations,  and  levels  of  intelligence. 
Interpersonal  skills  focus  more  on  social  skills;  it  has  been  clearly  demonstrated  that 
those  who  lack  interpersonal  skills  within  a  mono-cultural  situation  will  demonstrate  poor 
cross-cultural  skills  as  well  (see,  for  example,  Bochner,  2003).  Cultural  competence 
includes  culture  specific  knowledge,  such  as  proficiency  in  the  appropriate  language  and 
the  awareness  of  cultural  norms  and  taboos,  but  also  more  universal  abilities  such  as 
adapting  well  in  changing  circumstances  and  making  sense  of  unfamiliar  social 
environments  (Vaughn,  2010). 

This  study  proposes  a  related,  but  somewhat  more  focused,  perspective  on  the 
requirements  for  successful  cross-cultural  interactions.  While  a  number  of  cognitive  and 
emotional  skills  may  be  important,  to  varying  degrees,  to  one’s  knowledge  of  oneself,  it 
is  here  hypothesized  that  the  ability  to  successfully  navigate  the  differing  morality 
landscapes  of  other  cultures,  referred  to  here  as  moral  reasoning,  is  the  key  intrapersonal 
skill  underlying  cross-cultural  expertise.  This  study  will  measure  a  number  of 
intrapersonal  skills  and  characteristics,  but  the  focus  is  on  the  individual’s  capability  for 
moral  reasoning.  The  second  phase  of  this  study,  currently  under  development,  will 
propose  an  investigation  of  the  neural  bases  of  moral  reasoning,  in  part  to  determine  the 
relationship  between  the  brain  circuits  implicated  in  moral  reasoning  skills  and  those 
implicated  in  related  intrapersonal  capabilities  such  as  working  memory  and  cognitive 
control. 

In  terms  of  interpersonal  skills,  a  great  deal  of  attention  has  been  focused  over  the  past 
fifteen  years  on  social  mindreading,  which  generally  refers  to  an  individual’s  ability  to 
infer  the  internal  mental  states  (e.g.,  worried,  afraid,  puzzled)  of  another  person,  and  to 
use  that  inferred  state  to  predict  that  person’s  likely  actions.  Also  referred  to  in  the 
literature  as  “theory  of  mind”  or  “mentalizing,”  a  major  deficiency  in  social  mindreading 
skill  is  believed  to  lie  at  the  heart  of  the  social  deficits  of  those  suffering  from  autism 
spectrum  disorder  (Baron-Cohen,  1985;  Frith  &  Frith,  2006;  Apperly,  2012).  It  has  been 
postulated  that  mindreading  may  be  one  of  the  most  important,  as  well  as  one  of  the  most 
vulnerable,  of  all  interpersonal  social  skills  (see,  for  example,  Adolphs,  2010). 

Cultural  competence,  the  third  of  the  Vaughn  &  Phillips  triad,  is  not  as  easily  reduced  to 
a  specific  skill.  However,  early  work  by  Sieck  and  his  colleagues  (2008)  focused 
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attention  on  the  high-level  skill  of  sensemaking,  as  originally  proposed  by  Klein  and  his 
colleagues  (1993)  and  refined  by  Weick  (1995).  Sensemaking  is  the  idea  that  people 
interpret  a  situation  by  fitting  incoming  information  into  a  frame  that  links  them  to  other 
elements  (Klein,  Phillips,  Rail,  &  Peluso,  2004).  This  is  related  to  the  notion  of  cognitive 
flexibility  because  people  have  to  be  able  to  hold  many  frames  in  mind  and  test  these 
various  frames  against  incoming  information.  The  Sieck  focus  on  cultural  sensemaking 
is  a  natural  fit  to  the  cultural  competence  leg  of  the  triad  and  is  the  third  major  focus  of 
this  study. 

Previous  research  regarding  cross-cultural  adaptation  is  based  on  self-reported  data 
(either  interviews  or  questionnaires)  to  discover  so-called  “critical  components”  of 
culture-general  adaptability.  However,  self-report  studies  have  significant  risks  that  often 
make  the  results  more  variable  than  they  might  otherwise  be.  This  complicates  the  ability 
to  clearly  identify  the  underlying  factors  associated  with  success  or  failure,  and  may 
account  for  the  poor  agreement  between  these  studies  as  to  the  factors  that  are  the  most 
critical  for  cross-cultural  adaptability.  Based  on  the  multi-cultural  personality 
questionnaire  (van  der  Zee,  van  Oudenhoven,  &  de  Grijs,  2004),  college  students  scoring 
high  in  emotional  stability  and  cognitive  flexibility  reacted  more  positively  to  a  stressful 
intercultural  situation  relative  to  students  scoring  low  in  emotional  stability  and  cognitive 
flexibility.  In  a  study  of  peacekeepers,  Vanderpool  (2002)  reported  that  attention  to 
interpersonal  relations  and  being  open  to  new  experiences,  as  assessed  by  the  cross- 
cultural  adaptability  scale,  accounted  for  the  greatest  variability  in  their  achievement  of 
their  mission.  Ross  et  al.  (2010)  administered  the  cross-cultural  competence  inventory 
test  to  military  personnel  and  reported  that  cognitive  flexibility,  cross-cultural  empathy, 
openness,  and  a  willing  to  engage  with  other  cultures  were  the  key  factors  their  success 
with  challenge  requiring  cross  cultural  adaptation.  Although  the  specific  results  of  these 
three  studies  are  not  identical,  cognitive  flexibility  and  open-ness  were  commonly 
identified  as  key  to  cross  cultural  competence. 

In  contrast  to  self-report  metrics,  the  specific  psychological  instruments  and  tasks 
proposed  here  will  the  objective  assessment  of  cross-cultural  adaptability  while 
minimizing  those  problems  associated  with  self-report  data.  To  this  end,  we  intend  to 
administer  performance-based  tasks  to  experts  and  non-experts  in  cross-cultural 
adaptation,  as  well  as  self-report  questionnaires.  These  tasks,  along  with  the  necessary 
procedures  for  conducting  them  and  analyzing  the  resultant  data,  are  detained  in  the 
Study  Procedures  section  below 

This  protocol  constitutes  Phase  1  of  a  two  phase  study.  The  second  phase  will  build  on 
the  results  to  be  obtained  in  this  phase  by  utilizing  functional  brain  imaging  techniques  to 
explore  the  neural  bases  of  moral  reasoning  and  social  mindreading  as  they  relate  to 
cross-cultural  expertise. 

4.  Study  Procedures 
A.  Study  Design 
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1.  Participants 

All  participants  will  be  adults  between  18  and  60  years  of  age.  Some  of  the 
participants  will  be  recruited  by  the  research  project’s  sponsor  (the  Office  of 
Naval  Research,  or  ONR)  from  among  current  active-duty  military  personnel, 
while  others  may  have  significant  past  experience  in  military  operations  or 
government  foreign  service.  Both  male  and  female  participants  will  be  accepted, 
although  we  anticipate  having  fewer  females  volunteer  to  participate  because  of 
their  limited  numbers  within  the  military  and  ex-military  populations. 

In  this  first  phase  of  the  study,  data  will  be  collected  online  (i.e.,  over  the  Internet) 
from  two  types  of  volunteer  participants:  nominators  and  performers. 

•  A  nominator  is  a  participant  who  is  willing  to  nominate  one  or  more 
individuals  that  he/she  knows  well  and  whose  cross-cultural  skills  and 
experience  seem  to  be  a  good  match  for  this  study.  He/she  will  complete 
online  questionnaires  providing  justification  for  why  their  nominee(s) 
should  be  considered. 

•  A  performer  is  a  participant  that  fits  the  desired  set  of  cross-cultural  skills 
and  experience  for  this  study.  He/she  will  first  complete  an  online  survey 
in  which  their  particular  background  and  experience  in  cross-cultural 
interactions  will  be  documented.  The  perfonner  will  then  take  a  series  of 
behavioral  and  performance-based  psychological  tests  that  are  expected  to 
provide  quantitative  data  regarding  the  mental,  emotional,  and  social  skills 
of  cross-cultural  experts  and  non-experts. 

•  A  performer  does  not  need  to  first  be  nominated  in  order  to  participate. 
Likewise,  a  nominator  may  also  participate  as  a  performer,  if  he/she 
wishes.  The  purpose  of  the  nomination  process  is  to  provide  a  subjective 
third-party  perspective  of  a  given  performer’s  cross-cultural  skill  set,  and 
not  to  restrict  anyone  who  wishes  to  participate  as  a  performer  from  doing 
so. 


The  participants  will  take  these  questionnaires,  surveys,  and  tests  online  from 
remote  locations  of  their  own  choosing  via  a  website  created  by  JHU/APL 
specifically  for  this  project,  and  their  data  will  be  collected  and  stored  by  the 
research  team  at  JHU/APL.  Although  some  participants  may  live  or  work  in  the 
local  area,  it  is  anticipated  that  the  research  team  will  not  interact  face-to-face 
with  any  of  the  participants  during  this  phase. 

a.  Nominators 

Nominators  will  assist  the  categorization  of  the  performers  (as  described 
in  the  next  section)  by  providing  additional  insight  as  to  the  level  of  cross- 
cultural  expertise  possessed  by  the  person(s)  being  nominated.  The 
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nominator  will  be  asked  to  provide  an  email  address  for  each  person  being 
nominated;  the  website  will  then  automatically  generate  an  email  to  the 
nominee  inviting  them  to  consider  volunteering  for  the  study.  Nominators 
will  not  be  infonned  by  the  research  team  as  to  whether  or  not  their 
nominees  decided  to  participate.  Neither  the  submitted  email  addresses 
nor  any  of  the  collected  nomination  questionnaire  data  will  be  made 
available  to  anyone  outside  the  research  team  or  (upon  written  request)  the 
federal  research  sponsor  (ONR). 

b.  Performers 

Performers  will  be  recruited  to  fill  two  groups:  ‘experts’  and  ‘non¬ 
experts’.  The  decision  as  to  which  of  the  two  groups  a  given  perfonner  is 
to  be  placed  will  be  largely  subjective  (though  not  arbitrary).  Those 
performers  who  have  extensive  field  experience  in  cross-cultural 
interpersonal  situations  and  claim  to  have  solid  expertise  in  such 
interactions  will  be  considered  for  inclusion  in  the  ‘expert’  group,  as  will 
those  who  have  been  nominated  as  experts  by  their  peers  (i.e., 
nominators).  All  other  performers  will  be  assigned  to  the  ‘non-expert’ 
group. 

All  performers  will  take  the  same  tasks  during  the  course  of  the 
experiment.  The  actual  grouping  of  the  results  will  take  place  after  a 
given  performer’s  tests,  etc.,  have  been  completed  and  submitted,  and 
performers  will  not  be  told  into  which  group  their  results  will  be 
classified. 

2.  Recruitment 

a.  Active  Duty  Military  (ADM)  Participants 

Active  Duty  Military  participants  will  be  contacted  by  a  Military  Liaison 
Officer  (MLO)  designated  by  the  funder,  the  Office  of  Naval  Research,  as 
described  below.  Recruitment  will  remain  in  compliance  with  DoD 
Directive  3216.02,  “Protection  of  Human  Subjects  and  Adherence  to 
Ethical  Standards  in  DoD-Supported  Research”  dated  8  November  2011 
(further  discussed  on  the  JHM  IRB  website  at 
http://www.hopkinsmedicine.org/institutional_review_board/guidelines_p 
olicies/guidelines/dodreqs).  This  directive  requires  approval  from 
commanding  officers  before  active  duty  personnel  can  participate  in 
research  studies,  but  are  recruited  in  such  a  way  that  their  participation 
remains  voluntary.  Commanding  officers  are  not  informed  as  to  who  in 
their  command  has  participated  in  a  study. 
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The  specific  procedures  to  be  utilized  by  the  research  team  to  recruit  its 
participants  are  detailed  below. 

1 .  The  sponsor  of  this  research  (ONR)  will  designate  a  member  of  its 
organization  as  the  responsible  official  for  coordinating  and  facilitating  the 
recruitment  of  active-duty  military  personnel  (referred  to  here  as  the 
Military  Liaison  Officer,  or  MLO).  As  of  this  date,  ONR  has  designated 
Mr.  John  R.  Garvin  (ONR  Technology  Test  &  Demonstration  Office, 
Code  302)  as  the  MLO,  but  reserves  the  right  to  name  a  replacement  at  its 
own  discretion. 

2.  The  primary  responsibility  of  the  MLO  is  to  obtain  command 
approval  for  all  of  their  personnel  before  they  are  contacted  for  the 
research  study.  The  MLO  will  initially  contact  US  Marine  Corps  (USMC) 
operating  forces  headquarters,  such  as  Marine  Forces  Command  (the 
senior  operational  headquarters  for  all  US  Marine  Corps  operating  forces) 
and  the  USMC  Training  &  Education  Command,  to  obtain  command 
support  for  reaching  out  to  individual  marines  as  participants.  With  the 
concurrence  of  these  commands,  the  MLO  will  then  contact  additional 
subordinate  commands  as  necessary. 

3.  The  MLO  will  provide  each  organization  with  a  Research  Study 
Liaison  Document  that  provides: 

•  pertinent  details  and  constraints  of  the  study, 

•  the  operationally  relevant  benefits  of  participation, 

•  the  requirements  for  participation  in  the  study, 

•  assertions  that  any  data  collected  will  be  anonymous, 

•  assertions  that  the  individual  service  members’  respective  chains  of 
command  will  not  be  informed  who  participated  or  how  they  responded, 
and 

•  a  request  that  interested  participants  be  authorized  time  during  the 
training  day  to  complete  the  online  assessments  for  this  study. 

4.  If  a  given  organization  agrees  to  participate,  the  MLO  will  provide 
the  approved  recruiting  materials  (posters,  etc.)  that  list  the  website 
address.  If  desired,  the  organization  may  choose  to  designate  a  Unit 
Liaison  Officer  (ULO)  to  assist  with  the  recruiting  effort,  provided  that  the 
voluntary  participation  and  privacy  clauses  of  the  liaison  document  are 
inviolable. 

5.  No  one  outside  of  the  JHU/APL  research  team  will  be  aware  of 
who  has  or  has  not  elected  to  participate  in  the  research. 
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6.  Under  no  circumstances  will  JHU/APL  initiate  first  contact  with 
any  active-duty  military  personnel  who  have  not  been  nominated 
(nominees  are  automatically  contacted  via  email  by  the  website  software). 

b.  Non- ADM  Participants 

Recruitment  efforts  for  potential  volunteers  from  outside  of  the  active  duty 
military  community  will  utilize  more  conventional  methods;  specifically,  a 
targeted  community  outreach  effort  based  on  recruitment  flyers,  social 
network  advertisements,  etc..  The  same  recruitment  materials  will  be  used 
to  reach  out  to  all  potential  participants,  both  ADM  and  non-ADM.  The 
specific  forums  will  be  chosen  to  maximize  the  probability  of  reaching 
those  individuals  with  the  desired  cross-cultural  expertise. 

c.  JHU  APL  Participants 

JHU  APL  personnel  will  be  recruited  as  participants  for  the  associated 
pilot  study  that  will  develop  new  experiment  stimuli.  Recruitment  efforts 
for  APL  personnel  will  be  overseen  by  the  APL  Health  and  Environmental 
Services  (HES)  Department  and  will  comply  with  the  applicable  APL 
policies  and  procedures. 

3.  Experiment  Tasks 

a.  Nominators 

1.  Nomination  Questionnaire 

The  purpose  of  the  Nomination  Questionnaire  is  to  collect  third-party 
information  regarding  the  perceived  skill  level  of  a  nominated  (potential) 
performer.  This  questionnaire  focuses  on  the  nominator’s  opinions  regarding 
the  nominee’s  capabilities  in  real-world  cross-cultural  situations,  presumably 
gained  through  shared  experiences  or  personal  observation. 

2.  Ideal  Cross-Cultural  Expert  Questionnaire 

It  is  desirable  to  obtain  a  detailed  understanding  of  the  nominator’s 
perspectives  on  cross-cultural  expertise.  Consequently,  the  nominator  will 
also  be  requested  to  complete  a  questionnaire  in  which  he/she  is  asked  to 
characterize  the  capabilities  that  would  be  possessed  by  the  best  cross-cultural 
adapter  they  could  imagine. 

b.  Performers 
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There  are  a  number  of  standard  or  commonly-used  instruments  and  tasks 
available  to  quantitatively  examine  the  specific  psychological  skills  on  which  this 
study  focuses.  However,  it  is  impractical  to  include  all  of  the  desirable  measures 
if  the  total  time  being  asked  of  any  one  participant  is  to  be  limited  to  two  hours. 
Therefore,  the  list  of  performer  tasks  in  this  section  should  be  considered  a 
superset  of  the  ones  that  will  actually  be  deployed  on  the  website  for  the 
experiment.  The  choice  of  which  tasks  to  drop  will  be  made  during  the  final 
development  of  the  website,  balancing  the  information  to  be  gained  by  each  task 
with  the  time  requirements  and  difficulties  of  employing  the  respective  tasks  in  an 
online  system. 

1,  Demographic  and  Experience  Questionnaire 

We  will  collect  some  demographic  information  about  each  participant.  This 
will  include  information  like  age  and  gender,  but  also  information  about  any 
previous  deployments  and  how  often  they  have  engaged  in  activities  such  as 
traveling  abroad. 

2,  Cultural  Sensemaking  Task 

Sieck,  Smith,  &  Rasmussen  (2008)  found  that  a  critical  differentiating  feature 
between  experts  and  novices  in  cross-cultural  situations  was  the  ability  to 
engage  in  cultural  sensemaking.  This  ability  to  index  content,  make 
attributions  and  then  select  schema  is  a  key  differentiator  in  determining 
current  ability  at  cross-cultural  adaptability.  Sieck  and  Rasmussen  have  been 
developing  a  cultural  sensemaking  task  that  we  will  administer  to  our 
participants.  This  task  will  feature  about  24  different  vignettes  about  cross- 
cultural  situations  and  participants  will  have  to  answer  a  series  of  questions 
about  each  vignette. 

3,  Moral  Reasoning  Tasks 

As  noted  above,  moral  reasoning  capabilities  are  at  the  heart  of  this  study. 
The  tasks  listed  below  have  been  recommended  to  the  study  team  by  several 
of  the  most  reputable  researchers  that  specialize  in  this  area. 

a.  Moral  Foundations  Questionnaire  (MFQ) 

The  Moral  Foundations  Questionnaire  (Haidt  &  Graham,  2007;  Graham  et 
ah,  2011)  will  be  administered  to  participants.  This  30-item  assessment 
assesses  five  dimensions  of  moral  intuitions:  harm/care, 
fairness/reciprocity,  ingroup/loyalty,  authority/respect,  and  purity/sanctity. 
Responses  are  organized  on  a  Likert  scale. 
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b.  Condensed  Moral  Vignettes  (Knutson  et  al.,  2010) 

Selected  vignettes  representing  a  spectrum  of  challenge  will  be 
administered  ranging  from  compassionate  to  regretful,  sneaky, 
embarrassed  etc.  We  aim  to  include  challenges  that  are  positive,  neutral 
and  negative  in  valence.  Participants  are  asked  to  rate  (on  a  numbered 
anchored  rating  scale)  the  vignettes  along  the  following  dimensions: 
emotional  intensity,  emotion  aversion,  harm,  self-benefit,  other-benefit, 
pre-meditated,  illegality,  social  norm  violation  socialness,  social  norm 
violation  and  morally  appropriate. 

c.  Cognitive  Reflection  Test 

The  Cognitive  Reflection  Test  (CRT)  is  designed  to  assess  an  individual’s 
ability  to  suppress  a  spontaneous,  intuitive  (though  incorrect)  answer  in 
favor  of  a  more  reflective,  deliberative  (and  correct)  answer.  The 
participant  is  presented  with  a  series  of  “brainteasers”  to  be  answered  as 
quickly  and  as  accurately  as  possible.  It  is  generally  considered  to  place 
considerable  demand  on  the  individual’s  cognitive  control  mechanisms. 

d.  Need  for  Cognition  Scale 

The  Need  for  Cognition  Scale  quantitatively  measures  “the  tendency  for 
an  individual  to  engage  in  and  enjoy  thinking”  (Cacioppo  &  Petty,  1982, 
p.  116).  The  instrument  consists  of  18  statements.  The  participant 
responds  to  each  statement  using  a  9-point  scale  signifying  the  degree  to 
which  he/she  agrees  with  that  statement.  In  general,  the  instrument  takes 
10  minutes  or  less  to  complete. 

e.  Wisdom  Assessment 

The  Three  Dimensional  Wisdom  Scale  (3D-WS)  is  a  39-item  instrument 
that  examines  the  cognitive  (14  items),  reflective  (12  items),  and  affective 
(13  items)  dimensions  of  wisdom  (Ardelt,  2003).  Each  item  is  a  statement 
to  which  the  participant  responds  (on  a  5  point  scale)  how  much  he/she 
agrees  or  disagrees  with  the  statement.  Although  originally  created  for  use 
with  older  adults,  it  will  add  a  unique  dimension  to  the  assessments 
proposed  for  this  study. 

f.  Profile  of  Mood  States  (POMS  2-A  Short) 

The  Profile  of  Mood  States  (POMS)  (McNair  et  al.,  1981)  is  a  self-report 
scale  that  allows  for  the  quick  assessment  of  transient,  fluctuating  feelings, 
and  enduring  affect  states.  The  tool  is  applicable  in  clinical,  medical, 
research,  and  athletic  settings  and  is  sensitive  to  change.  The  POMS 
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consists  of  seven  subscales:  Anger-Hostility,  Confusion-Bewilderment, 
Depression-Dejection,  Fatigue-Inertia,  Tension-Anxiety,  Vigor- Activity, 
Friendliness. 


g.  ERQ  (Gross  &  John,  2003) 

ERQ  is  a  10-item  measure  of  emotion  regulation  strategies  on  a  7-point 
scale  (1  =  strongly  disagree  and  7  =  strongly  agree),  yielding  two 
independent  subscales  for  cognitive  reappraisal  and  expressive 
suppression  with  adequate  reliability.  Frequent  reappraisal,  compared  with 
frequent  suppression,  is  associated  with  greater  positive  mood,  social 
functioning,  and  psychological  well-being  and  lower  negative  mood 
(including  depressive  symptoms)  compared  with  frequent  suppression. 

h.  Emotional  Stroop  Task 

The  Emotional  Stroop  test  (Gotlib  et  ah,  1984)  is  a  task  designed  as  an 
information-processing  approach  to  assessing  emotions.  Related  to  the 
standard  Stroop  effect,  the  Emotional  Stroop  test  examines  the  response 
time  of  the  participant  to  name  colors  of  negative  emotional  words.  The 
Emotional  Stroop  assesses  attention  and  response  time  due  to  the 
emotional  relevance  of  the  word  for  the  individual  (Algom  et  ah,  2005). 
The  Emotional  Stroop  test  has  been  used  broadly  in  clinical  studies  using 
emotional  words  related  to  a  particular  individual's  area  of  concern,  such 
as  alcohol-related  words  for  someone  who  is  alcoholic,  or  words  involving 
a  particular  phobia  for  someone  with  anxiety  or  phobic  disorders. 

i.  Mood  Awareness  Scale 

The  Mood  Awareness  Scale  was  created  by  Swinkels  &  Gulianoto  (1995) 
to  evaluate  an  individual’s  ability  to  assess  his/her  own  mood.  Research 
findings  suggest  that,  compared  with  low  mood  monitors,  high  mood 
monitors  show  greater  self-consciousness,  are  more  neurotic,  have  lower 
self-esteem,  and  experience  greater  negative  affect.  Compared  with  low 
mood  labelers,  high  mood  labelers  tend  to  be  less  socially  anxious,  less 
neurotic,  more  extraverted,  and  more  nonverbally  expressive,  and  to 
experience  greater  positive  affect  (Swinkels  &  Guilianoto,  1995). 

4.  Mindreading  Tasks 

The  following  tasks  are  under  consideration  for  inclusion  in  this  study.  While 
there  are  many  different  mindreading  tasks  in  the  published  literature,  four 
have  been  identified  as  the  most  suitable  for  the  goals  of  this  study.  Because 
of  the  excessive  time  that  would  be  required  to  have  all  of  the  participants 
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perform  all  four  of  these  tasks,  it  is  unlikely  that  all  four  will  be  utilized  in  this 
phase  of  the  research.  Current  plans  call  for  two  of  the  four  to  be  selected 
based  on  the  availability  of  stimuli,  pilot  data,  and  recommendations  from 
outside  experts. 

The  four  tasks  are  described  in  more  detail  below. 


a.  The  Strange  Stories  Task  (Happe,  1994;  Jolliffe  &  Baron-Cohen, 
1999) 

A  set  of  24  short  vignettes,  each  accompanied  by  a  picture  and  two  test 
questions,  is  presented  to  participants.  The  vignettes  concern  everyday 
situations  where  people  say  things  that  they  do  not  mean  literally.  The 
story  types  are  classified  as:  lie,  white  lie,  joke,  pretend,  misunder¬ 
standing,  persuade,  appearance/reality,  figure  of  speech,  sarcasm,  forget, 
double  bluff,  and  contrary  emotions.  Six  additional  stories  are  used  as 
controls  and  involve  unforeseen  outcomes  with  a  mechanical  or  physical 
cause. 

b.  Visual  Joke  Appreciation  Task  (Corcoran,  Cahill,  &  Frith,  1997) 

Two  sets  of  10  visual  comics  or  jokes  are  presented  to  participants  who 
must  explain  why  the  jokes  are  funny.  One  set  of  jokes  involves  slapstick 
or  behavioral  humor,  while  the  other  set  requires  an  understanding  of  the 
main  character’s  mental  state 

c.  Reading  the  Mind  in  the  Eyes  Task  (Baron-Cohen  et  al.,  1997;  Baron- 
Cohen  et  al.,  2001;  Adams  et  al.,  2010) 

The  participant  is  presented  with  a  series  of  36  photographs  of  the  eye 
region  of  the  face  of  different  individuals,  and  is  asked  to  choose  which  of 
four  words  best  describes  what  the  person  in  the  photograph  is  thinking  or 
feeling  (e.g.,  “serious”,  “ashamed”,  “alarmed”,  “bewildered”).  Some  of 
the  faces  will  be  of  Caucasian  Americans,  and  others  of  different  cultural 
groups  (e.g.,  Turks). 

d.  Non-verbal  Attribution  of  Intention  Task  (Brunet  et  al.,  2000;  Brunet 
et  al.,  2003) 

In  this  task,  the  participant  is  presented  with  a  series  of  three-panel  comic 
strips  that  convey  a  vignette  about  an  individual.  The  participant  is  then 
given  three  alternative  pictures  that  serve  as  potential  endings  to  the 
vignette.  In  order  to  select  the  correct  picture,  the  participant  will  need  to 
correctly  infer  the  character’s  intention  in  the  first  three  panels. 
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5.  Psychological  Assessments 

The  tasks  described  below  are  focused  on  specific  mental  characteristics  that 
will  be  used  to  supplement  the  findings  from  the  cultural  sensemaking,  moral 
reasoning,  and  social  mindreading  tasks  above.  If  the  study’s  hypothesis  is 
correct,  then  the  test  results  from  the  tasks  in  this  section  will  either  be  highly 
correlated  with  the  previous  tasks,  or  will  account  for  relatively  little  of  the 
variance  between  the  two  groups. 

a.  Intelligence  Assessment 


We  will  administer  two  to  three  tasks  from  the  Manual  for  Kit  of  Factor- 
Referenced  Cognitive  Tests  ((Ekstrom,  French,  Harman,  &  Derman,  1976; 
Roberts  et  ah,  2000)  to  our  participants  as  a  way  to  estimate  general  fluid 
intelligence  (GFI).  These  cognitive  tests  have  previously  been  applied  to  a 
military  population,  and  have  been  broken  down  into  different  factors  to 
allow  us  to  better  determine  certain  cognitive  factors  we  think  most 
relevant.  We  will  focus  on  tasks  that  relate  to  induction  ability  and 
integrative  processes,  two  key  components  of  both  general  fluid 
intelligence  and  of  sensemaking.  The  score  of  GFI  will  be  used  as  a 
covariate  to  control  for  other  factors  to  allow  us  to  better  determine  what 
explains  difference  in  ability  beyond  any  differences  in  intelligence 
ability.  If  GFI  is  sufficient  to  explain  performance  in  cross-cultural 
adaptability,  this  would  be  an  important  finding.  It  generally  takes  about  7 
minutes  to  do  each  task,  so  should  take  about  20  minutes  or  less  to  do  all 
the  tasks  we  administer. 


b.  Operation  Span 

The  operation  span  task,  or  OSPAN  (Turner  &  Engle,  1989)  is  a  test  of 
working  memory  span  that  alternates  between  doing  simple  mathematical 
tasks  and  reading  letters,  followed  by  a  recall  task  of  the  letters  presented. 
This  task  correlates  with  Raven’s  (Unsworth  &  Engle,  2005),  so  can  be 
viewed  as  a  supplement  to  the  IQ  test.  It  takes  about  twelve  minutes  to 
conduct  this  test. 

c.  NEO-FFI 

The  Neuroticism-Extroversion-Openness  Five-Factor  Inventory  or  NEO- 
FFI  (Costa  &  McCrae,  1992)  is  a  60-question  five-factor  personality 
assessment.  Of  particular  interest  to  us  are  questions  relating  to  openness, 
as  previous  research  has  sometimes  shown  that  openness  to  new 
experiences  and  new  ideas  correlates  with  cross-cultural  skills  (Ross  et  al., 
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2010;  Vanderpool,  2002).  We  hypothesize  that  experts  will  score  higher 
on  tests  of  openness  than  will  novices. 

d.  MPQ-40 

The  short  form  of  the  multicultural  personality  questionnaire  (van  der  Zee, 
van  Oudenhoven,  Ponterotto,  &  Fietzer,  2013)  is  40  questions  with  five 
subscales;  cultural  empathy,  emotional  stability,  social  initiative,  open- 
mindedness,  and  flexibility.  The  MPQ  predicts  variability  in  cross- 
cultural  behavioral  competence  beyond  the  big  5  personality  traits.  We 
expect  to  find  experts  score  higher  on  all  subscales  than  do  non-experts. 

e.  IRI 

The  interpersonal  reactivity  index  (Davis,  1983)  is  a  14-item  self-report 
that  measures  empathy.  Although  we  will  administer  the  whole  test,  we 
expect  that  two  of  the  subscales  in  particular  will  differentiate  experts 
from  non-experts:  perspective-taking  and  empathic  concern.  We  expect 
that  these  seven  items  will  load  higher  for  experts  than  for  non-experts. 

f.  EIS 

The  emotional  intelligence  scale  (Schutte  et  ah,  1998)  is  a  33-item 
questionnaire  measuring  emotional  intelligence,  appraisal  and  regulation. 
Questions  are  rated  on  a  five-point  Likert  scale.  We  expect  that  experts 
will  score  higher  on  this  scale  than  will  non-experts. 

g.  CQS 

The  20-item  cultural  intelligence  scale  (CQS)  is  meant  to  predict  an 
individual’s  capacity  to  perfonn  effectively  in  cross-cultural  situations 
(Ang  et  ah,  2007).  We  expect,  as  previously  found  by  Abbe  &  Bortnick 
(2010),  that  those  with  more  experience  and  more  expertise  will  score 
higher  in  CQS.  The  subscales  of  the  CQ  Scale  (CQS)  assess  four 
dimensions;  metacognitive  CQ,  cognitive  CQ,  motivational  CQ  and 
behavioral  CQ. 

4.  Overview  of  Study  Procedures 

Most  of  the  study  tasks  and  associated  stimuli  for  this  experiment  are  based  on 
standard  psychology  instruments  or  on  tasks  that  have  already  been  published  in 
the  open  scientific  literature.  However,  in  a  few  key  cases  it  will  be  necessary  to 
prepare  and  validate  new  stimuli  in  order  to  better  achieve  the  objectives  of  the 
research.  Procedures  for  developing  and  validating  new  stimuli  via  initial  pilot 
studies  are  described  in  the  first  section  below. 
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The  remainder  of  this  section  covers  the  procedures  to  be  followed  during  the 
actual  performance  of  the  study.  This  study  relies  heavily  on  a  set  of  web  pages 
and  associated  software  to  administer  the  consent  procedures,  manage  the 
nomination  process,  link  the  data  supplied  by  nominators  to  their  nominees  (for 
those  nominees  that  volunteer  to  be  performers),  administer  the  experiment  tasks 
and  questionnaires,  and  collect  the  data  for  subsequent  analysis.  The  procedures 
to  be  employed  by  the  web  pages  and  followed  by  participants  are  described 
below. 


a.  Pilot  Studies  of  New  Stimuli 

1.  New  “Reading  the  Mind  in  the  Eyes”  Stimuli 

New  ‘eyes’  stimuli  will  be  prepared  and  piloted  utilizing  a  similar 
procedure  to  the  one  used  during  the  development  of  the  original 
stimuli  (Baron-Cohen,  et  ah,  2001). 

a.  Generation  of  New  Stimuli 

Fifty  images  of  faces  from  the  identified  culture/race  will  be 
selected  from  publically-available  sources  (e.g.,  magazines,  the 
Internet,  etc.)  and  then  cropped  and  reformatted  to  comply  with  the 
original  grey-scale,  500  x  200  pixel  format  of  the  original  stimuli. 
Target  words  and  foils  will  then  be  generated  by  members  of  the 
research  team.  The  target  words  and  foils  will  be  selected  from  the 
list  used  in  the  original  research  (Appendix  A  of  Baron-Cohen,  et 
al„  2001). 

b.  Pilot  Study 

Teams  consisting  of  eight  evaluators  will  be  used  to  pilot  test  the 
new  stimuli.  In  order  for  a  given  stimulus  image  to  be  rated  as 
acceptable,  both  of  the  following  two  conditions  must  be  met: 

1)  at  least  five  out  of  the  eight  evaluators  must  select  the  same 
target  word,  and 

2)  no  more  than  two  of  the  remaining  evaluators  can  select  the 
same  one  of  the  three  available  foils. 

Items  that  fail  to  meet  both  of  these  requirements  will  either  be 
discarded  or  reworked  with  new  target  words,  foils,  or  both. 
Reworked  stimuli  will  be  re-piloted  by  a  new  group  of  evaluators 
until  they  meet  both  of  the  requirements  listed  above. 
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After  successfully  passing  the  evaluator  teams’  testing,  the  stimuli 
will  be  piloted  with  a  larger  group  (N  =  40)  of  volunteer  graders 
recruited  from  within  APL.  The  stimuli  will  be  presented  via  a 
custom  web  site  and  evaluated  by  the  graders  online.  In  keeping 
with  the  goals  of  this  research,  an  effort  will  be  made  to  recruit 
volunteer  graders  from  diverse  ethnic  and  cultural  backgrounds.  In 
order  for  a  given  stimulus  image  to  be  rated  as  acceptable,  both  of 
the  following  two  conditions  must  be  met: 

1)  at  least  50%  of  the  graders  must  select  the  target  word,  and 

2)  no  more  than  25%  of  the  graders  can  select  the  same  one  of  the 
three  available  foils. 

At  the  conclusion  of  the  pilot,  36  of  the  piloted  stimuli  from  within 
each  of  the  two  novel  racial/cultural  groups  will  be  selected  on  the 
basis  of  their  pilot  scores.  This  will  yield  a  final  experiment 
stimulus  set  of  144  stimuli:  36  each  from  the  Caucasian,  African- 
American,  Arab,  and  Oriental  groups. 

b.  Data  Collection  Software.  Data  collection  for  this  study  will  be  done 
using  Qualtrics  (qualtrics.com),  a  well-established  online  survey 
administration  service.  JHUAPL  owns  a  license  and  uses  this  service 
frequently.  Study  PI  Jonathon  Kopecky  and  study  team  members 
Jason  Spitaletta  and  Nathan  Bos  have  access  to  the  account  at  this 
time;  other  study  team  members  will  be  given  accounts  only  if  needed. 
Respondents  login  to  the  system  with  their  email  address.  (Passwords 
would  not  offer  any  added  security  benefit  in  this  system.  While  it  is 
possible  that  another  person  could  log  in  with  a  respondent's  email, 
this  would  not  allow  them  to  access  prior  responses.)  Data  sent  to  and 
from  Qualtrics  is  encrypted  with  SSL. 

Within  the  system,  participants'  email  is  is  kept  in  one  file  (internally 
referred  to  as  a  'panel')  linked  to  an  ID  number.  The  ID  is  the  only 
identifier  used  in  the  survey  response  files.  Because  biographical  data 
on  deployments  and  overseas  work  is  included  in  both  the  Nominator 
and  Respondent  surveys,  this  data  is  considered  to  contain  indirect 
identifying  information,  and  is  handled  accordingly. 

Qualtrics'  statement  on  data  security  is  available  on  request. 
Qualtrics  conducts  background  checks  on  employees  and  limits  access 
to  customer  data  to  a  small  set  of  database  administration  personnel. 

c.  Web  Site  Registration  Procedures 
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This  protocol  is  based  on  the  concept  of  remote  (conceivably  from  all  over 
the  country)  volunteers  logging  into  a  web  site  and  completing 
questionnaires  and  tests  without  any  need  for  them  to  interact  directly  with 
the  research  team.  In  order  to  make  this  system  work,  a  Qualtrics  web  site 
will  be  created  by  JHU/APL  to  support  the  necessary  experiment 
procedures,  consent  processes,  and  data  collection,  as  well  as  to  provide 
mechanisms  for  protecting  participant  privacy  (e.g.,  through  the  automated 
association  and  de-identification  of  data). 

The  web  site  will  use  a  participant’s  email  address  as  a  user  name,  and 
will  link  the  various  data  from  multiple  sources  (e.g.,  from  multiple 
nominations  and  the  participant’s  own  performance  data)  using  the  email 
address  as  the  key.  Nominators  will  be  required  to  provide  an  email 
address  for  their  nominees  (see  section  [c]  below),  and  performers  will 
register  using  their  email  address  as  described  in  this  section.  It  should  be 
noted  that  the  participant’s  name  and  email  address  will  not  appear  in  any 
of  the  data  supplied  by  the  system;  the  system  will  assign  a  random 
identifier  (e.g.,  ‘p-54’)  to  each  participant  which  will  replace  the  email 
address  in  the  database  provided  to  the  research  team. 

All  potential  participants  must  register  prior  to  utilizing  the  website.  The 
registration  process  is  described  below. 

1.  Upon  reaching  the  home  page  of  the  web  site,  the  user  will  see  a  brief 
description  of  the  study’s  purpose  and  a  list  of  the  inclusion 
requirements  for  study  participants.  There  will  be  a  statement 
encouraging  those  who  meet  the  criteria  to  register  (if  this  is  the  first 
time  visiting  the  site)  or  log  in  (if  already  registered)  to  the  site. 

2.  Registration  will  require  the  user  to  provide  his/her  email  address.  In 
addition,  a  random  non-machine  readable  word  will  be  displayed, 
which  the  user  must  type  into  a  response  block  in  order  to  prevent 
automated  algorithms  (robots)  from  entering  the  site. 

3.  To  ensure  that  the  email  provided  by  the  user  is  a  workable  address,  an 
email  will  be  automatically  sent  to  the  provided  email  address  that 
contains  a  link  for  finalizing  the  registration.  When  the  user  logs  into 
his/her  email  account,  reads  the  email,  and  selects  the  confirmation 
link,  he/she  will  be  registered  into  the  database  as  a  registered  system 
user. 

4.  Once  the  user  has  successfully  logged  into  the  system,  he/she  will  be 
transferred  to  a  subordinate  web  page  that  explains  the  two  roles  for 
participants:  ‘nominator’  and  ‘performer.’  Selecting  a  particular  role 
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will  then  transfer  the  user  to  the  portion  of  the  site  dedicated  to  that 
role,  as  described  in  sections  [d]  and  [e]  below. 

5.  Registered  users  may  log  out  of  the  site  and  then  return  at  a  later  time 
or  date  as  necessary  or  convenient.  However,  once  a  ‘performer’  has 
completed  the  questionnaires  and  experiment  tasks  associated  with  that 
role,  he/she  will  be  unable  to  return  to  the  ‘performer’  pages  of  the  site 
for  the  duration  of  the  experiment. 

d.  Nominator  Procedures 

A  participant  that  wishes  to  nominate  one  or  more  individuals  with  whom 
he/she  is  familiar  may  do  so  by  volunteering  as  a  nominator.  The 
nomination  process  will  be  handled  completely  by  the  Qualtrics  software 
within  the  JHU/APL  website,  as  described  in  more  detail  below  and 
summarized  in  Figure  4-1.  A  participant  may  elect  to  withdraw  his/her 
consent  to  participate  at  any  time  up  to  the  final  data  submission  (step  5 
below),  and  all  nomination  materials  (names,  emails,  questionnaire  results, 
etc.)  will  be  excluded  from  any  analyses. 

1 .  A  participant  who  wishes  to  nominate  one  or  more  individuals  will  log 
into  their  account  on  the  JHU/APL  website  (if  not  already  logged  on). 
He/she  will  then  select  the  “Nominator”  role. 

2.  Once  in  the  “Nominator”  section,  the  participant  will  be  provided  an 
electronic  consent  form.  If  the  participant  understands  the  information 
and  agrees  to  continue  his/her  participation,  he/she  may  move  on  to  the 
nomination  process.  If  the  participant  has  questions  or  concerns  about 
the  consent  document,  then  he/she  will  be  directed  to  send  an  email  to 
the  PI  to  obtain  clarification,  and  will  be  permitted  to  logout  and  return 
after  the  questions  or  concerns  have  been  answered.  If  the  participant 
declines  to  consent  to  the  form,  then  the  session  will  be  cancelled  and 
the  participant  logged  off. 
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Figure  4- 1 :  Nominator  Procedure 

3.  If  the  participant  consents  to  continue,  then  he/she  will  be  transferred 
to  the  nominations  page.  On  this  page,  the  participant  will  record  the 
name  and  email  address  of  one  or  more  nominees.  (The  email  address 
will  be  used  internally  by  the  software  application  to  keep  track  of  the 
nominees  and  to  account  for  multiple  nominations  of  the  same 
individual.)  For  each  nominee,  the  participant  will  be  asked  to  answer 
a  set  of  opinion  questions  regarding  the  social  skills  and  traits  of  the 
nominee  (attached).  When  finished  with  each  nominee  and  related 
opinion  questions,  the  participant  will  submit  the  information  and  can 
then  nominate  an  additional  individual  or  move  on  to  the  remainder  of 
the  nomination  task. 
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4.  The  software  application  will  then  check  the  email  address  of  the 
nominee  against  the  master  database  it  maintains  of  all  registered  users 
of  the  system.  If  the  email  address  does  not  match  any  registered  user, 
the  system  will  automatically  generate  and  send  an  email  to  the 
nominee  informing  them  that  they  have  been  nominated  and  providing 
recruitment  materials  to  encourage  them  to  participate  in  the 
experiment. 

5.  After  the  last  performer  candidate  has  been  nominated,  the  participant 
will  be  asked  to  complete  a  short  questionnaire  that  explores  what 
skills  and  traits,  in  general,  make  up  (at  least,  in  the  participant’s 
opinion)  the  ideal  cross-cultural  expert.  When  the  questionnaire  has 
been  completed,  the  participant  will  submit  the  data  and  move  to  the 
next  page. 

6.  Finally,  the  participant  will  be  offered  a  chance  to  participate  in  the 
experiment  as  a  performer  as  well.  If  he/she  elects  to  do  so,  the  system 
will  shift  them  to  the  “Performer”  section  as  described  in  the  next 
section.  Otherwise,  the  participant  will  be  thanked  for  his/her 
participation  and  will  be  logged  off. 

e.  Performer  Procedures 

A  participant  that  wishes  to  volunteer  as  a  performer  will  follow  the 
procedure  that  is  described  in  more  detail  below  and  summarized  in  Figure 
4-2.  As  was  the  case  for  the  nomination  process,  the  performer  process 
will  be  handled  completely  by  the  Qualtrics  software  within  the  JHU/APL 
website.  A  performer  may  elect  to  withdraw  his/her  consent  to  participate 
at  any  time  up  to  the  final  data  submission  and  all  experiment  data  (task 
results,  questionnaire  results,  etc.)  will  be  excluded  from  any  analyses. 

1.  A  participant  who  has  volunteered  to  perform  the  experiment  tasks 
will  log  into  their  account  on  the  JHU/APL  website  (if  not  already 
logged  on).  He/she  will  then  select  the  “Performer”  role. 

2.  Once  in  the  “Performer”  section,  the  participant  will  be  provided  an 
electronic  consent  form  (attached).  If  the  participant  understands  the 
information  and  agrees  to  continue  his/her  participation,  he/she  may 
move  on  to  the  nomination  process.  If  the  participant  has  questions  or 
concerns  about  the  consent  document,  then  he/she  will  be  directed  to 
send  an  email  to  the  PI  to  obtain  clarification,  and  will  be  permitted  to 
logout  and  return  after  the  questions  or  concerns  have  been  answered. 
If  the  participant  declines  to  consent  to  the  form,  then  the  session  will 
be  cancelled  and  the  participant  logged  off. 
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3.  If  the  participant  consents  to  continue,  then  he/she  will  proceed  to  the 
background  and  experience  survey  page.  On  this  page,  the  participant 
will  be  asked  to  answer  a  set  of  survey  questions  regarding  his/her 
prior  experience  in  social  situations  requiring  cross-cultural 
adaptability  (attached). 

4.  After  completing  the  background  and  experience  survey,  the 
participant  will  move  on  to  the  set  of  experiment  tasks  described  in 
Section  4.A.3,  “Experiment  Tasks,”  above.  It  is  anticipated  that 
completing  these  tasks  will  require  the  bulk  of  the  participant’s  time 
and  effort  that  is  dedicated  to  this  research  effort.  Although  the  tasks 
have  been  selected  so  that  the  entire  effort  can  be  completed  in  a  single 
session  of  two  hours  or  less,  the  participant  will  be  given  the  option  of 
saving  his  work  at  specific  points  during  the  testing  and  then 
continuing  later  during  a  separate  session. 


Figure  4-2:  Performer  procedure 
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5.  Finally,  the  participant  will  be  asked  to  submit  the  results  using  a 
button  on  the  web  page.  Once  the  performer  questionnaire  has  been 
completed,  the  participant  will  be  locked  out  of  the  “performer”  mode 
during  any  future  sessions,  with  the  intention  of  ensuring  that  only  a 
single  set  of  data  will  be  collected  from  any  given  performer. 


B.  Study  duration  and  number  of  study  visits  required  of  research  participants. 

The  first  phase  of  this  study  (the  focus  of  this  protocol)  is  expected  to  be  completed 
by  the  end  of  this  calendar  year  (2013).  The  associated  data  collection  will  occur 
over  a  maximum  period  of  three  months. 

Participants  will  not  ‘visit’  any  research  facility  per  se,  as  the  experiment  will 
conducted  entirely  online  via  the  Internet.  The  participant  may  visit  the  web  site  as 
frequently  as  desired  during  a  two  week  period  following  initial  registration,  after 
which  the  participant  will  no  longer  be  allowed  to  access  the  site.  Participants  who 
volunteer  as  ‘performers’  will  be  allowed  a  maximum  of  one  week  to  complete  the 
associated  questionnaires  and  performance  tasks. 

C.  Blinding,  including  justification  for  blinding  or  not  blinding  the  trial,  if 
applicable. 

Data  collection  will  be  performed  entirely  via  the  JHU/APL  web  site.  No  face-to- 
face  interaction  between  the  research  team  and  the  participants  is  envisioned.  Any 
subjective  evaluation  of  the  experiment  task  data  (e.g.,  judging  the  acceptability  of 
participant  responses  in  some  of  the  ‘mindreading’  tasks)  will  be  perfonned  after  the 
data  has  been  de-identified.  Consequently,  no  need  is  seen  for  blinding  of  the 
experiment. 

D.  Justification  of  why  participants  will  not  receive  routine  care  or  will  have 
current  therapy  stopped. 

N/A 

E.  Justification  for  inclusion  of  a  placebo  or  non-treatment  group. 

N/A 

F.  Definition  of  treatment  failure  or  participant  removal  criteria. 

Participants  will  be  removed  from  the  study  if  they  fail  to  complete  the  perfonner 
tasks  within  one  week  of  commencing  them. 
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G.  Description  of  what  happens  to  participants  receiving  therapy  when  study  ends 
or  if  a  participant’s  participation  in  the  study  ends  prematurely. 

N/A 

5.  Inclusion/Exclusion  Criteria 

All  participants  will  be  adults,  between  18  and  60  years  of  age.  An  effort  will  be  made  to 
recruit  both  male  and  female  participants. 

For  the  pilot  studies,  all  participants  will  also  be  JHU/APL  employees. 

For  the  main  experiment,  additional  inclusion  criteria  for  all  participants  are: 

1.  have  access  to  the  Internet,  the  World  Wide  Web,  and  a  personal  email  account, 
and 

2.  have  the  basic  computer  skills  necessary  to  navigate  a  straightforward  web  site 
and  perform  the  basic  data  entry  activities  commonly  encountered  on  such  sites 
(e.g.,  creating  a  password,  clicking  on  action  buttons,  and  entering  freeform  text). 

For  the  main  experiment,  additional  inclusion  criteria  for  performer  participants  are: 

1 .  must  have  traveled  outside  of  the  country  at  least  once  in  the  last  five  years 

Volunteers  with  valid  email  addresses  and  whose  computer  skills  are  sufficient  to  allow 
them  access  to  the  experiment’s  web  site  will  not  be  prevented  from  performing  the  tasks 
associated  with  the  experiment.  However,  the  data  collected  from  any  participant  whose 
questionnaires  or  test  score  indicate  that  the  individual  does  not  meet  the  inclusion 
criteria  listed  above  will  be  discarded  prior  to  analysis  and  will  not  be  incorporated  into 
the  final  results. 

6.  Drugs/  Substances/  Devices 

No  investigational  drugs,  substances,  or  devices  will  be  utilized  in  this  phase  of  the  study. 

a.  The  rationale  for  choosing  the  drug  and  dose  or  for  choosing  the  device  to  be 
used. 

N/A 

b.  Justification  and  safety  information  if  FDA  approved  drugs  will  be  administered 
for  non-FDA  approved  indications  or  if  doses  or  routes  of  administration  or 
participant  populations  are  changed. 

N/A 
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c.  Justification  and  safety  information  if  non-FDA  approved  drugs  without  an  IND 
will  be  administered. 

N/A 

7.  Study  Statistics 

This  study  will  ask  participants  to  perform  several  standard  psychology  instruments  (e.g., 
IQ  tests)  and  a  set  of  related  performance  tests  (e.g.,  the  cultural  sensemaking  task).  The 
participants  will  all  take  the  same  battery  of  tests,  but  will  be  divided  into  two  groups 
(namely,  “expert”  and  “non-expert”)  for  the  purpose  of  data  analysis.  Consequently,  this 
is  a  between-subjects  design  in  which  the  outcome  differences  are  assumed  to  result  in 
the  differences  between  the  ‘treatments’  or  ‘manipulations’  given  to  the  two  groups.  In 
the  case,  the  ‘treatment’  for  this  study  is  the  subjective  level  of  cross-cultural  expertise 
possessed  by  each  group,  necessitating  that  the  participants  be  selected  for  a  given  group 
based  on  their  backgrounds  and  experience,  rather  than  the  random  assignment  to  groups 
that  is  usually  prescribed  for  a  between-subjects  experiment. 

As  a  result,  this  study  design  is  not  a  true  experiment,  in  which  the  treatment  is 
completely  controlled  by  the  experimenters,  but  is  instead  a  correlational  study  based  on 
the  classification  variable  ‘cross-cultural  expertise,’  or  CCE,  which  substitutes  here  for 
the  more  common  independent  variable  of  a  true  experiment. 

a.  Primary  outcome  variable 

The  primary  outcome  variable  is  a  vector-valued  quantity  that  has  been  designated 
“cross-cultural  potential”  or  CCP,  as  described  below. 

This  study  will  collect  a  set  of  scores  from  the  selected  instruments  and  tasks  that  are 
performed  by  each  performer  participant.  Some  of  the  test  scores  will  probably  be 
correlated  to  each  other  to  varying  degrees,  and  some  may  demonstrate  little  correlation 
to  the  classification  variable.  The  collection  of  test  scores  from  each  participant  will  be 
used  to  create  a  vector-valued  variable,  named  “Test  Scores.”  Factor  analysis  will  be 
performed  to  create  a  transformation  from  the  Test  Score  vector  to  a  lower-dimensional 
vector,  the  CCP.  The  correlation  between  CCP  and  the  classification  variable  CCE  will 
then  be  calculated  using  multiple  regression  and  correlation  techniques. 

b.  Secondary  outcome  variables 

No  analyses  of  secondary  outcome  variables  are  planned. 

c.  Statistical  plan  including  sample  size  justification  and  interim  data  analysis 

The  plan  is  to  test  a  minimum  of  150  participants.  A  power  analysis  was  conducted  using 
the  G*Power  application  (version  3.1.6,  ©  2013  by  Franz  Faul,  Universtaet  Kiel, 
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Germany;  http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/)  for  a 
multiple  regression  and  assuming  the  following  parameters: 

•  effect  size  =  0.6  (recommended  by  Cohen,  1988,  as  a  medium  effect  size) 

•  alpha  =  0.05 

•  power  (1  -  beta)  =  0.80 

•  number  of  predictors  =  8  (the  result  was  fairly  insensitive  to  this  value) 

The  power  analysis  resulted  in  a  minimum  sample  size  of  105  participants.  Due  to  the 
minimal  risk  nature  of  the  study  and  the  lack  of  monetary  compensation  being  offered  to 
the  participants,  the  costs  of  collecting  a  larger  sample  size  are  negligible.  Consequently, 
a  minimum  sample  size  of  150  participants  is  recommended,  though  a  larger  sample  size 
should  be  obtained  if  practicable  within  the  time  limits  and  the  nature  of  the  sought-after 
population. 

d.  Early  stopping  rules. 

Due  to  the  minimal  risk  nature  of  the  protocol,  no  early  stopping  rule  is  necessary. 
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8.  Risks 

There  are  no  risks  to  participants  that  are  above  minimal  risk. 

a.  Medical  risks,  listing  all  procedures,  their  major  and  minor  risks  and  expected 
frequency 

There  are  no  medical  risks  associated  with  this  procedure  above  what  would  be 
normally  encountered  in  every-day  life  situations. 

b.  Steps  taken  to  minimize  the  risks 

N/A 

c.  Plan  for  reporting  unanticipated  problems  or  study  deviations 

If  unanticipated  problems  occur,  or  deviations  from  the  approved  protocol  are  noted 
or  reported  by  any  of  the  research  staff,  the  collection  of  data  (if  ongoing)  will  be 
suspended,  the  JHM  IRB  will  be  notified  of  the  discrepancy  or  problem,  and  the  grant 
sponsor  (ONR)  will  be  informed.  Data  collection  will  not  resume  until  corrective 
measures  have  been  identified,  the  protocol  updated/resubmitted/reapproved  as 
necessary,  and  the  permission  of  the  JHM  IRB  to  recommence  the  study  has  been 
obtained. 

d.  Legal  risks  such  as  the  risks  that  would  be  associated  with  breach  of 
confidentiality 

The  data  to  be  collected  will  be  de-identified  prior  to  being  analyzed  by  the  research 
team.  The  risk  of  a  breach  of  confidentiality  has  been  minimized  to  the  extent 
practicable. 

e.  Financial  risks  to  the  participants 

There  are  no  known  or  anticipated  financial  risks  for  the  participants. 

9.  Benefits 

There  are  no  immediate  benefits  to  the  participants  from  this  research. 

The  potential  benefits  to  society  are  many.  This  basic  science  effort  to  understand  the 
bases  of  intercultural  adaptation  is  intended  to  lead  to  more  applied  studies  focused 
on  effective  methods  for  teaching  people  of  all  cultures  how  to  adapt  rapidly  and 
effectively  to  each  other’s  cultural  environment.  Society,  now  more  than  ever,  is 
dependent  on  the  close  cooperation  between  people  from  different  cultures  and 
nationalities  to  achieve  the  benefits  of  shared  opportunities,  capabilities  and  resources 
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while  avoiding  unnecessary  conflict  and  violence.  In  particular,  by  focusing  on 
military  personnel  who  have  served  in  foreign  countries,  this  study  would  quite 
naturally  lead  to  greater  opportunities  for  civil-military  cooperation  across  cultural 
boundaries  and  lessen  the  likelihood  of  resorting  to  violence  to  achieve  national  aims. 
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10.  Payment  and  Remuneration 

Participants  in  the  pilot  studies  for  the  development  of  new  stimuli  will  be  APL 
employees  who  are  by  APL  policies  prohibited  from  accepting  additional  monetary 
payment  from  such  participation. 

Participants  in  the  main  experiment  (both  nominators  and  performers)  will  not  be  paid 
or  remunerated  for  their  efforts  under  this  protocol.  Likewise,  no  penalties  will  be 
assessed  for  any  participant  who  fails  to  the  complete  the  protocol  for  any  reason. 

11.  Costs 

There  are  no  costs  to  participants  for  participation  in  this  effort  other  than  the 
investment  of  their  time,  which  is  anticipated  to  be  two  hours  or  less  for  performers 
and  considerably  less  for  nominators.  The  costs  associated  with  the  development  and 
support  of  the  experiment  website  will  be  paid  for  by  the  research  team. 
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